Re: addfile() for create new nc file

From: gibies george <gibiesgeorge_at_nyahnyahspammersnyahnyah>
Date: Wed Aug 17 2011 - 07:53:11 MDT

Thank you for your reply.

On 17 August 2011 18:54, Mary Haley <haley@ucar.edu> wrote:

> Gibies,
>
> The reason you are getting a bunch of \0 characters is because in order to
> write a string array to a NetCDF file, it has to be converted to a character
> array. If your strings are not the same length, then they will be padding
> with "end-of-string" characters, which are the \0 that you see.
>
> Writing 2D arrays of strings is not allowed in NetCDF-3 (and I believe in
> NetCDF-4), so what you're doing is not standard.
>
> I don't think there will be any way to avoid the \0 characters. I suppose
> you can just write your strings out as one big string. That is, if you have:
>
> mld1 = "ond_ssn"
> mld2 = "cfsv2_jul_init"
> mld3 = "cfsv2_aug_init"
>
> then instead of making a 2D character array out of these, first concatenate
> these all into one big string:
>
> mld_big_string = mld1 + " " + mld2 + " " + mld3
>
> and then convert mld_big_string to a 1D character array:
>
> multi_model_data@mdl = stringtocharacter(mld_big_string)
>
> This should then look like the following on the file:
>
> multi_model_data:mld = "ond_ssn cfsv2_jul_init cfsv2_aug_init\0"
>
> --Mary
>
>
>
> On Aug 17, 2011, at 1:54 AM, gibies george wrote:
>
>
>
> On 16 August 2011 22:52, Mary Haley <haley@ucar.edu> wrote:
>
>> I don't understand something here. You have a "multi_model_data" variable
>> that has an array of different units. Does this mean you are trying to
>> write different variables all to the same array, and then use one of the
>> dimensions to indicate the number of different variables you've written to
>> the one array?
>>
>> If this is what you are trying to accomplish, I'm not sure I understand
>> why. Why not write each variable separately to the file?
>>
>
> I write them together because it is the first mode of combined-EOF of
> different parameters (wind850_rain_mslp_tsfc) of seasonal
> mean inter-annual variability.
>
> But my problem is more or less technical about writing array of string to
> data variable attribute to a file.
>
> Here is the output from ncdump.
>
> multi_model_data:mdl =
> "ond_ssn\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000cfsv2_jul_init\000\000\000\000\000\000\000\000\000cfsv2_aug_init\000\000\000\000\000\000\000\000\000ECMWF_aug_init\000\000\000\000\000\000\000\000\000IFM_Geomer_aug_init\000\000\000\000Meteo_France_aug_init\000\000UK_MetOffice_aug_init\000\000CMCC_INGV_aug_init\000\000\000\000\000DePreSys_UKMO_aug_init"
> ;
>
> More specifically why is this \000\000\000\000\000\000\000\000\000\000\000\
> patter appearing and how can I avoid that ?
>
>
>
>
>>
>> --Mary
>>
>>
>> multidata!6 ="mdl"
>>> multidata&mdl = ispan(1,nofmdls,1)
>>> multidata@mdl = mdl
>>>
>>
>>>
>>> On Aug 15, 2011, at 1:23 AM, gibies george wrote:
>>
>>
>>
>> On 10 August 2011 13:59, gibies george <gibies@tropmet.res.in> wrote:
>>
>>> Thank you Dennis and Mary for your support and suggestions.
>>>
>>> My problem is solved by doing a slight variation in my script according
>>> to your suggestions. Some how I had to retain the string information and so
>>> I made them to be attributes to the data variable.
>>>
>>
>> But again there is a problem in writing the attributes. Here is the output
>> of ncdump
>>
>> netcdf ond_ssn_multimodel_data {
>> dimensions:
>> time = 25 ;
>> lat = 180 ;
>> lon = 360 ;
>> season = 1 ;
>> component = 6 ;
>> ens = 1 ;
>> mdl = 9 ;
>> variables:
>> float multi_model_data(time, lat, lon, season, component, ens, mdl) ;
>> multi_model_data:unit =
>> "m/s\000\000\000\000m/s\000\000\000\000m/s\000\000\000\000mm/day\000hPa\000\000\000\000degC"
>> ;
>> multi_model_data:units =
>> "m/s\000\000\000\000m/s\000\000\000\000m/s\000\000\000\000mm/day\000hPa\000\000\000\000degC"
>> ;
>> multi_model_data:region = "global" ;
>> multi_model_data:ssnend = 12 ;
>> multi_model_data:ssnbegin = 10 ;
>> multi_model_data:dataset = "ond_ssn_1981_to_2005" ;
>> multi_model_data:datavar = "wind850_rain_mslp_tsfc" ;
>> multi_model_data:anal_start_year = 1981 ;
>> multi_model_data:anal_end_year = 2005 ;
>> multi_model_data:latsouth = -90 ;
>> multi_model_data:latnorth = 90 ;
>> multi_model_data:lonwest = 0 ;
>> multi_model_data:loneast = 360 ;
>> multi_model_data:mpCenterLonF = 180 ;
>> multi_model_data:novctr = 1 ;
>> multi_model_data:nosclr = 3 ;
>> multi_model_data:long_name = "wind850_rain_mslp_tsfc" ;
>> multi_model_data:ens = "ond_ssn" ;
>> multi_model_data:mdl =
>> "ond_ssn\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000cfsv2_jul_init\000\000\000\000\000\000\000\000\000cfsv2_aug_init\000\000\000\000\000\000\000\000\000ECMWF_aug_init\000\000\000\000\000\000\000\000\000IFM_Geomer_aug_init\000\000\000\000Meteo_France_aug_init\000\000UK_MetOffice_aug_init\000\000CMCC_INGV_aug_init\000\000\000\000\000DePreSys_UKMO_aug_init"
>> ;
>> multi_model_data:component =
>> "wind850_u\000\000\000\000\000wind850_v\000\000\000\000\000wind850_speed\000rain\000\000\000\000\000\000\000\000\000\000mslp\000\000\000\000\000\000\000\000\000\000tsfc"
>> ;
>> multi_model_data:season = "ond" ;
>> multi_model_data:_FillValue = -9.96921e+36f ;
>> int time(time) ;
>> time:_FillValue = -999 ;
>> float lat(lat) ;
>> float lon(lon) ;
>> int season(season) ;
>> season:ssnbegin = 10 ;
>> season:ssnend = 12 ;
>> int component(component) ;
>> int ens(ens) ;
>> int mdl(mdl) ;
>>
>> Can anybody suggest a solution.
>>
>>
>>
>>> But I still feel it is better to have more direct way to do all these
>>> things. Please inform me if there is some simpler way.
>>>
>>> Thanking you once again
>>>
>>>
>>> *Here is my modified script.*
>>>
>>> ________________________________________________________________________________________________
>>>
>>> multidata =
>>> new((/dimsizes(time),dimsizes(lat),dimsizes(lon),dimsizes(season),dimsizes(component),\
>>> nofens,nofmdls/),float)
>>> multidata!0 = "time"
>>> multidata&time = time
>>> multidata!1 = "lat"
>>> multidata&lat = lat
>>> multidata!2 = "lon"
>>> multidata&lon = lon
>>> multidata!3 ="season"
>>> multidata&season=ind(.not.ismissing(season))
>>> multidata@season = season
>>> multidata!4 ="component"
>>> multidata&component=ind(.not.ismissing(component))
>>> multidata@component = component
>>> multidata!5 ="ens"
>>> multidata&ens = ispan(1,nofens,1)
>>> multidata@ens = ens
>>> multidata!6 ="mdl"
>>> multidata&mdl = ispan(1,nofmdls,1)
>>> multidata@mdl = mdl
>>> -------------------
>>> -----------
>>> ----------------------
>>> ----------
>>> multidata@dataset = mdl@analdset + "_" + predef@anal_start_year + "_to_"
>>> + predef@anal_end_year
>>> multidata@datavar = datavar
>>> multidata&ens = ind(.not.ismissing(ens))
>>> multidata&mdl = ind(.not.ismissing(mdl))
>>> print("---------------Multimodel Ensemble-------------------")
>>> print("_____________________________________________________")
>>> printVarSummary(multidata)
>>> print("_____________________________________________________")
>>>
>>> syout= systemfunc("rm " + mdl@analdset+"_multimodel_data.nc")
>>> outfile = addfile(mdl@analdset+"_multimodel_data.nc","c")
>>> outfile->multi_model_data = multidata
>>> print("Data is written. Press any key to continue.")
>>> response = systemfunc("read var; echo $var")
>>>
>>> ___________________________________________________________________________________________
>>>
>>> *
>>> *
>>> *Here is the output.*
>>>
>>> Variable: multidata
>>> Type: float
>>> Total Size: 12960000 bytes
>>> 3240000 values
>>> Number of Dimensions: 7
>>> Dimensions and sizes: [time | 25] x [lat | 180] x [lon | 360] x [season
>>> | 1] x [component | 1] x [ens | 2] x [mdl | 1]
>>> Coordinates:
>>> time: [1981..2005]
>>> lat: [-90..90]
>>> lon: [ 0..360]
>>> season: [0..0]
>>> component: [0..0]
>>> ens: [0..1]
>>> mdl: [0..0]
>>> Number Of Attributes: 22
>>> unit : mm/day
>>> units : mm/day
>>> long_name : _rain
>>> nosclr : 1
>>> novctr : 0
>>> mpCenterLonF : 180
>>> loneast : 360
>>> lonwest : 0
>>> latnorth : 90
>>> latsouth : -90
>>> anal_end_year : 2005
>>> anal_start_year : 1981
>>> datavar : _rain
>>> dataset : cfsv2_jjas_1981_to_2005
>>> ssnbegin : 6
>>> ssnend : 9
>>> region : global
>>> mdl : cfsv2
>>> ens : ( feb_initial , may_initial )
>>> component : rain
>>> season : jjas
>>> _FillValue : -9.96921e+36
>>> (0) _____________________________________________________
>>> (0) Data is written. Press any key to continue.
>>>
>>>
>>> ____________________________________________________________________________________________
>>>
>>>
>>>
>>>
>>> On 6 June 2011 21:53, Mary Haley <haley@ucar.edu> wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> NetCDF doesn't allow you to have strings for coordinate arrays. You must
>>>> convert these to 2D character arrays.
>>>>
>>>> Please see the "Coordinate Systems" section of this NetCDF document:
>>>>
>>>> http://www.unidata.ucar.edu/software/netcdf/docs/BestPractices.html
>>>>
>>>> I don't know how you created "season" in your code below, but you can
>>>> try converting it to a character array, and use this as your coordinate
>>>> variable:
>>>>
>>>> cseason = tocharacter(season)
>>>> multidata&season = cseason
>>>>
>>>> --Mary
>>>>
>>>> On Jun 6, 2011, at 1:11 AM, gibies george wrote:
>>>>
>>>> Dear NCL users,
>>>>
>>>> Somebody please suggest a way to eliminate the following warning
>>>> messages.
>>>>
>>>> I was trying to write my dataset to an nc file.
>>>>
>>>> Thanking you in davance.
>>>>
>>>> *_______________________________________
>>>> ________________Script__________________
>>>> _______________________________________*
>>>> -------------------------
>>>> -----------------------
>>>> ----------------------------
>>>> ---------------------
>>>> --------------------------
>>>>
>>>> multidata@dataset = mdl@analdset + "_" + predef@anal_start_year+ "_to_" + predef@anal_end_year
>>>> multidata@datavar = datavar
>>>> multidata&ens = ens
>>>> multidata&mdl = mdl
>>>> print("---------------Multimodel Ensemble-------------------")
>>>> print("_____________________________________________________")
>>>> printVarSummary(multidata)
>>>> print("_____________________________________________________")
>>>>
>>>> syout= systemfunc("rm " + mdl@analdset+"_multimodel_data.nc")
>>>> outfile = addfile(mdl@analdset+"_multimodel_data.nc","c")
>>>> outfile->multi_model_data = multidata
>>>>
>>>> return(multidata)
>>>> end ;;; End of function definition get_multimodel_data() ;;;
>>>>
>>>> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>>>> *_______________________________________*
>>>>
>>>> *_________________________________
>>>> ___________ Output _______________
>>>> _________________________________*
>>>>
>>>>
>>>> (0) ---------------Multimodel Ensemble-------------------
>>>> (0) _____________________________________________________
>>>>
>>>>
>>>> Variable: multidata
>>>> Type: float
>>>> Total Size: 272160000 bytes
>>>> 68040000 values
>>>> Number of Dimensions: 7
>>>> Dimensions and sizes: [time | 25] x [lat | 280] x [lon | 360] x
>>>> [season | 1] x [component | 3] x [ens | 1] x [mdl | 9]
>>>> Coordinates:
>>>> time: [1981..2005]
>>>> lat: [-35..35]
>>>> lon: [30..120]
>>>> season: [ond..ond]
>>>> component: [rain..tsfc]
>>>> ens: [..]
>>>> mdl: [cfsv2_jun_init..DePreSys_UKMO_aug_init]
>>>> Number Of Attributes: 18
>>>> unit : ( mm/day, hPa, degC )
>>>> units : ( mm/day, hPa, degC )
>>>> region : indotropic
>>>> ssnend : 12
>>>> ssnbegin : 10
>>>> dataset : ond_skill_1981_to_2005
>>>> datavar : _rain_mslp_tsfc
>>>> anal_start_year : 1981
>>>> anal_end_year : 2005
>>>> latsouth : -35
>>>> latnorth : 35
>>>> lonwest : 30
>>>> loneast : 120
>>>> mpCenterLonF : 75
>>>> novctr : 0
>>>> nosclr : 3
>>>> long_name : _rain_mslp_tsfc
>>>> _FillValue : -9.96921e+36
>>>> (0) _____________________________________________________
>>>> fatal:Attempting to write variable (season) of type (string) which is
>>>> not representable in the format of file (ond_skill_multimodel_data.nc)
>>>> fatal:Trying using a type conversion function
>>>> warning:FileWriteVarVar: Could not write coordinate variable (season) to
>>>> file (ond_skill_multimodel_data), continuing anyway
>>>> fatal:Attempting to write variable (component) of type (string) which is
>>>> not representable in the format of file (ond_skill_multimodel_data.nc)
>>>> fatal:Trying using a type conversion function
>>>> warning:FileWriteVarVar: Could not write coordinate variable (component)
>>>> to file (ond_skill_multimodel_data), continuing anyway
>>>> fatal:Attempting to write variable (ens) of type (string) which is not
>>>> representable in the format of file (ond_skill_multimodel_data.nc)
>>>> fatal:Trying using a type conversion function
>>>> warning:FileWriteVarVar: Could not write coordinate variable (ens) to
>>>> file (ond_skill_multimodel_data), continuing anyway
>>>> fatal:Attempting to write variable (mdl) of type (string) which is not
>>>> representable in the format of file (ond_skill_multimodel_data.nc)
>>>> fatal:Trying using a type conversion function
>>>> warning:FileWriteVarVar: Could not write coordinate variable (mdl) to
>>>> file (ond_skill_multimodel_data), continuing anyway
>>>> warning:Execute: Error occurred at or near line 3188 in file
>>>> /home/gibies/ncl/mylib/lib_gibies_read_data_module.ncl
>>>>
>>>>
>>>>
>>>> --
>>>> *Gibies George, CSIR-RF,*
>>>> *Monsoon Seasonal Prediction Group,
>>>> Climate and Global Modelling Division,
>>>> Indian Institute of Tropical Meteorology,
>>>> Dr. Homi Bhabha Road,
>>>> NCL (P. O.), Pashan,
>>>> Pune 411008, India.*
>>>>
>>>> *http://sites.google.com/site/gibiesge/*
>>>>
>>>> Please Think about the environment. Save paper; Save Trees; and don't
>>>> print this e-mail unless it is necessary.
>>>> _______________________________________________
>>>> ncl-talk mailing list
>>>> List instructions, subscriber options, unsubscribe:
>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Gibies George, CSIR-RF,*
>>> *Monsoon Seasonal Prediction Group,
>>> Climate and Global Modelling Division,
>>> Indian Institute of Tropical Meteorology,
>>> Dr. Homi Bhabha Road,
>>> NCL (P. O.), Pashan,
>>> Pune 411008, India.*
>>>
>>> *http://sites.google.com/site/gibiesge/*
>>>
>>> Please Think about the environment. Save paper; Save Trees; and don't
>>> print this e-mail unless it is necessary.
>>>
>>
>>
>>
>> --
>> *Gibies George, CSIR-RF,*
>> *Monsoon Seasonal Prediction Group,
>> Climate and Global Modelling Division,
>> Indian Institute of Tropical Meteorology,
>> Dr. Homi Bhabha Road,
>> NCL (P. O.), Pashan,
>> Pune 411008, India.*
>>
>> *http://sites.google.com/site/gibiesge/*
>>
>> Please Think about the environment. Save paper; Save Trees; and don't
>> print this e-mail unless it is necessary.
>>
>>
>>
>
>
> --
> *
> *
> *
> Gibies George, CSIR-RF,
> **Climate and Global Modelling Division,
> Indian Institute of Tropical Meteorology,
> Dr. Homi Bhabha Road,
> NCL (P. O.), Pashan,
> Pune 411008, India.
> Office mail : gibies@tropmet.res.in
> Mobile: 9881707796 (at Pune)*
>
> *http://sites.google.com/site/gibiesge/*
>
> Please Think about the environment. Save paper; Save Trees; and don't print
> this e-mail unless it is necessary.
> *
> *
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>

-- 
*
*
*
Gibies George,   CSIR-RF,
**Climate and Global Modelling Division,
Indian Institute of Tropical Meteorology,
Dr. Homi Bhabha Road,
NCL (P. O.), Pashan,
Pune 411008, India.
Office mail : gibies@tropmet.res.in
Mobile: 9881707796 (at Pune)*
*http://sites.google.com/site/gibiesge/*
Please Think about the environment. Save paper; Save Trees; and don't print
this e-mail unless it is necessary.
*
*

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Wed Aug 17 07:53:26 2011

This archive was generated by hypermail 2.1.8 : Mon Aug 22 2011 - 08:13:37 MDT