Re: Error in concatenation of multiple files

On Jun 24, 2013, at 11:33 AM, Abhik Santra wrote:

> Dear Mary,
>
> Thanks for your reply.
> My data files are very long time-series and I need a part of it only.
> So, reading the whole time-series, as you have suggested in earlier email, is not a good option as my PC's memory is not supporting that!!
> Is there any other way to solve the problem?

I realize now that you said each file has one *level* of data, so I'm not sure why you want to concatenate the data based on the time array.

What part do you need? You need to be more clear about how you are subsetting the data, and what you need to eventually do with it.

If you are getting an error because of memory, it would help if you could include the exact error. We might be able to help with this.

If the level is different for each data array, I would think you'd want a new "level" dimension (and hence use the "join" option rather than the "cat" option), rather than concatenating across time. But, this would only work if the time values are the same on each file:

latS = -10.
latN = 10.
files = systemfunc ("ls -1rt /data/Anomaly/*.nc")
fall = addfiles (files, "r")
ListSetType (fall, "join")

X = fall[:]->xanom(:,{latS:latN},:)
printVarSummary(X)

This should give you a 12 x 4018 x 20 x 360 array, where the "12" represents the number of files (and hence the levels), and the 4018 represents time.
As I said, though, this will only work if the times are the same on each file.

--Mary

> I am using ndtooned and onedtond functions afterward to rearrange the input array.
> Thank you once again.
>
> Regards,
> Abhik
>
>
> ----- Original Message -----
> From: "Mary Haley" <haley@ucar.edu>
> To: "Abhik Santra" <abhiksantra@tropmet.res.in>
> Cc: "ncl-talk" <ncl-talk@ucar.edu>
> Sent: Monday, June 24, 2013 8:22:30 PM
> Subject: Re: Error in concatenation of multiple files
>
>
> On Jun 24, 2013, at 8:19 AM, Abhik Santra wrote:
>
>> Hi,
>>
>> I am trying to read multiple files (12 files), each having dimensions (ntim|4018 x nlat|20 x mlon|360).
>> Each file contains the data at a particular vertical level (total 12 levels).
>>
>> My code is the following:
>>
>> latS = -10.
>> latN = 10.
>> files = systemfunc ("ls -1rt /data/Anomaly/*.nc")
>> fall = addfiles (files, "r")
>> ListSetType (fall, "cat")
>>
>> X = fall[:]->xanom(0:4017,{latS:latN},:)
>> printVarSummary(X)
>>
>> I should get the dimension of X as 48216 x 20 x 360.
>> But the printVarSummary output for variable X is:
>>
>> Variable: X
>> Type: float
>> Total Size: 115718400 bytes
>> 28929600 values
>> Number of Dimensions: 3
>> Dimensions and sizes: [time | 4018] x [lat | 20] x [lon | 360]
>> Coordinates:
>> time: [ 0.. 4017]
>> lat: [-9.5.. 9.5]
>> lon: [ 0.. 359]
>>
>> It appears that files are not merged properly so far.
>> Can you please correct me if I have made any mistakes?
>> Thanks in advance.
>
> You are subscripting the leftmost (time) dimension to be 0:4017, which is why you are getting 4018 points:
>
>> X = fall[:]->xanom(0:4017,{latS:latN},:)
>
> Try
>
>> X = fall[:]->xanom(:,{latS:latN},:)
>
>
>>
>> Regards,
>> Abhik
>>
>> ___________________________________________
>>
>> Abhik Santra
>> CSIR Research Fellow,
>> Indian Institute of Tropical Meteorology,
>> Pune - 411008.
>> India
>> ___________________________________________
>> _______________________________________________
>> ncl-talk mailing list
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
> --
> ___________________________________________
>
> Abhik Santra
> CSIR Research Fellow,
> Indian Institute of Tropical Meteorology,
> Pune - 411008.
> India
> ___________________________________________
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Mon Jun 24 12:57:51 2013