Re: addfile() for create new nc file

From: Mary Haley <haley_at_nyahnyahspammersnyahnyah>
Date: Wed Aug 17 2011 - 07:24:45 MDT

Gibies,

The reason you are getting a bunch of \0 characters is because in order to write a string array to a NetCDF file, it has to be converted to a character array. If your strings are not the same length, then they will be padding with "end-of-string" characters, which are the \0 that you see.

Writing 2D arrays of strings is not allowed in NetCDF-3 (and I believe in NetCDF-4), so what you're doing is not standard.

I don't think there will be any way to avoid the \0 characters. I suppose you can just write your strings out as one big string. That is, if you have:

mld1 = "ond_ssn"
mld2 = "cfsv2_jul_init"
mld3 = "cfsv2_aug_init"

then instead of making a 2D character array out of these, first concatenate these all into one big string:

   mld_big_string = mld1 + " " + mld2 + " " + mld3

and then convert mld_big_string to a 1D character array:

  multi_model_data@mdl = stringtocharacter(mld_big_string)

This should then look like the following on the file:

multi_model_data:mld = "ond_ssn cfsv2_jul_init cfsv2_aug_init\0"

--Mary

On Aug 17, 2011, at 1:54 AM, gibies george wrote:

>
>
> On 16 August 2011 22:52, Mary Haley <haley@ucar.edu> wrote:
> I don't understand something here. You have a "multi_model_data" variable that has an array of different units. Does this mean you are trying to write different variables all to the same array, and then use one of the dimensions to indicate the number of different variables you've written to the one array?
>
> If this is what you are trying to accomplish, I'm not sure I understand why. Why not write each variable separately to the file?
>
> I write them together because it is the first mode of combined-EOF of different parameters (wind850_rain_mslp_tsfc) of seasonal mean inter-annual variability.
>
> But my problem is more or less technical about writing array of string to data variable attribute to a file.
>
> Here is the output from ncdump.
>> multi_model_data:mdl = "ond_ssn\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000cfsv2_jul_init\000\000\000\000\000\000\000\000\000cfsv2_aug_init\000\000\000\000\000\000\000\000\000ECMWF_aug_init\000\000\000\000\000\000\000\000\000IFM_Geomer_aug_init\000\000\000\000Meteo_France_aug_init\000\000UK_MetOffice_aug_init\000\000CMCC_INGV_aug_init\000\000\000\000\000DePreSys_UKMO_aug_init" ;
>
> More specifically why is this \000\000\000\000\000\000\000\000\000\000\000\
> patter appearing and how can I avoid that ?
>
>
>
>
>
> --Mary
>
>
>> multidata!6 ="mdl"
>> multidata&mdl = ispan(1,nofmdls,1)
>> multidata@mdl = mdl
>
>
> On Aug 15, 2011, at 1:23 AM, gibies george wrote:
>
>>
>>
>> On 10 August 2011 13:59, gibies george <gibies@tropmet.res.in> wrote:
>> Thank you Dennis and Mary for your support and suggestions.
>>
>> My problem is solved by doing a slight variation in my script according to your suggestions. Some how I had to retain the string information and so I made them to be attributes to the data variable.
>>
>> But again there is a problem in writing the attributes. Here is the output of ncdump
>>
>> netcdf ond_ssn_multimodel_data {
>> dimensions:
>> time = 25 ;
>> lat = 180 ;
>> lon = 360 ;
>> season = 1 ;
>> component = 6 ;
>> ens = 1 ;
>> mdl = 9 ;
>> variables:
>> float multi_model_data(time, lat, lon, season, component, ens, mdl) ;
>> multi_model_data:unit = "m/s\000\000\000\000m/s\000\000\000\000m/s\000\000\000\000mm/day\000hPa\000\000\000\000degC" ;
>> multi_model_data:units = "m/s\000\000\000\000m/s\000\000\000\000m/s\000\000\000\000mm/day\000hPa\000\000\000\000degC" ;
>> multi_model_data:region = "global" ;
>> multi_model_data:ssnend = 12 ;
>> multi_model_data:ssnbegin = 10 ;
>> multi_model_data:dataset = "ond_ssn_1981_to_2005" ;
>> multi_model_data:datavar = "wind850_rain_mslp_tsfc" ;
>> multi_model_data:anal_start_year = 1981 ;
>> multi_model_data:anal_end_year = 2005 ;
>> multi_model_data:latsouth = -90 ;
>> multi_model_data:latnorth = 90 ;
>> multi_model_data:lonwest = 0 ;
>> multi_model_data:loneast = 360 ;
>> multi_model_data:mpCenterLonF = 180 ;
>> multi_model_data:novctr = 1 ;
>> multi_model_data:nosclr = 3 ;
>> multi_model_data:long_name = "wind850_rain_mslp_tsfc" ;
>> multi_model_data:ens = "ond_ssn" ;
>> multi_model_data:mdl = "ond_ssn\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000cfsv2_jul_init\000\000\000\000\000\000\000\000\000cfsv2_aug_init\000\000\000\000\000\000\000\000\000ECMWF_aug_init\000\000\000\000\000\000\000\000\000IFM_Geomer_aug_init\000\000\000\000Meteo_France_aug_init\000\000UK_MetOffice_aug_init\000\000CMCC_INGV_aug_init\000\000\000\000\000DePreSys_UKMO_aug_init" ;
>> multi_model_data:component = "wind850_u\000\000\000\000\000wind850_v\000\000\000\000\000wind850_speed\000rain\000\000\000\000\000\000\000\000\000\000mslp\000\000\000\000\000\000\000\000\000\000tsfc" ;
>> multi_model_data:season = "ond" ;
>> multi_model_data:_FillValue = -9.96921e+36f ;
>> int time(time) ;
>> time:_FillValue = -999 ;
>> float lat(lat) ;
>> float lon(lon) ;
>> int season(season) ;
>> season:ssnbegin = 10 ;
>> season:ssnend = 12 ;
>> int component(component) ;
>> int ens(ens) ;
>> int mdl(mdl) ;
>>
>> Can anybody suggest a solution.
>>
>>
>> But I still feel it is better to have more direct way to do all these things. Please inform me if there is some simpler way.
>>
>> Thanking you once again
>>
>>
>> Here is my modified script.
>> ________________________________________________________________________________________________
>>
>> multidata = new((/dimsizes(time),dimsizes(lat),dimsizes(lon),dimsizes(season),dimsizes(component),\
>> nofens,nofmdls/),float)
>> multidata!0 = "time"
>> multidata&time = time
>> multidata!1 = "lat"
>> multidata&lat = lat
>> multidata!2 = "lon"
>> multidata&lon = lon
>> multidata!3 ="season"
>> multidata&season=ind(.not.ismissing(season))
>> multidata@season = season
>> multidata!4 ="component"
>> multidata&component=ind(.not.ismissing(component))
>> multidata@component = component
>> multidata!5 ="ens"
>> multidata&ens = ispan(1,nofens,1)
>> multidata@ens = ens
>> multidata!6 ="mdl"
>> multidata&mdl = ispan(1,nofmdls,1)
>> multidata@mdl = mdl
>> -------------------
>> -----------
>> ----------------------
>> ----------
>> multidata@dataset = mdl@analdset + "_" + predef@anal_start_year + "_to_" + predef@anal_end_year
>> multidata@datavar = datavar
>> multidata&ens = ind(.not.ismissing(ens))
>> multidata&mdl = ind(.not.ismissing(mdl))
>> print("---------------Multimodel Ensemble-------------------")
>> print("_____________________________________________________")
>> printVarSummary(multidata)
>> print("_____________________________________________________")
>>
>> syout= systemfunc("rm " + mdl@analdset+"_multimodel_data.nc")
>> outfile = addfile(mdl@analdset+"_multimodel_data.nc","c")
>> outfile->multi_model_data = multidata
>> print("Data is written. Press any key to continue.")
>> response = systemfunc("read var; echo $var")
>> ___________________________________________________________________________________________
>>
>>
>> Here is the output.
>>
>> Variable: multidata
>> Type: float
>> Total Size: 12960000 bytes
>> 3240000 values
>> Number of Dimensions: 7
>> Dimensions and sizes: [time | 25] x [lat | 180] x [lon | 360] x [season | 1] x [component | 1] x [ens | 2] x [mdl | 1]
>> Coordinates:
>> time: [1981..2005]
>> lat: [-90..90]
>> lon: [ 0..360]
>> season: [0..0]
>> component: [0..0]
>> ens: [0..1]
>> mdl: [0..0]
>> Number Of Attributes: 22
>> unit : mm/day
>> units : mm/day
>> long_name : _rain
>> nosclr : 1
>> novctr : 0
>> mpCenterLonF : 180
>> loneast : 360
>> lonwest : 0
>> latnorth : 90
>> latsouth : -90
>> anal_end_year : 2005
>> anal_start_year : 1981
>> datavar : _rain
>> dataset : cfsv2_jjas_1981_to_2005
>> ssnbegin : 6
>> ssnend : 9
>> region : global
>> mdl : cfsv2
>> ens : ( feb_initial , may_initial )
>> component : rain
>> season : jjas
>> _FillValue : -9.96921e+36
>> (0) _____________________________________________________
>> (0) Data is written. Press any key to continue.
>>
>> ____________________________________________________________________________________________
>>
>>
>>
>>
>> On 6 June 2011 21:53, Mary Haley <haley@ucar.edu> wrote:
>>
>> Hi,
>>
>> NetCDF doesn't allow you to have strings for coordinate arrays. You must convert these to 2D character arrays.
>>
>> Please see the "Coordinate Systems" section of this NetCDF document:
>>
>> http://www.unidata.ucar.edu/software/netcdf/docs/BestPractices.html
>>
>> I don't know how you created "season" in your code below, but you can try converting it to a character array, and use this as your coordinate variable:
>>
>> cseason = tocharacter(season)
>> multidata&season = cseason
>>
>> --Mary
>>
>> On Jun 6, 2011, at 1:11 AM, gibies george wrote:
>>
>>> Dear NCL users,
>>>
>>> Somebody please suggest a way to eliminate the following warning messages.
>>>
>>> I was trying to write my dataset to an nc file.
>>>
>>> Thanking you in davance.
>>>
>>> _______________________________________
>>> ________________Script__________________
>>> _______________________________________
>>> -------------------------
>>> -----------------------
>>> ----------------------------
>>> ---------------------
>>> --------------------------
>>>
>>> multidata@dataset = mdl@analdset + "_" + predef@anal_start_year + "_to_" + predef@anal_end_year
>>> multidata@datavar = datavar
>>> multidata&ens = ens
>>> multidata&mdl = mdl
>>> print("---------------Multimodel Ensemble-------------------")
>>> print("_____________________________________________________")
>>> printVarSummary(multidata)
>>> print("_____________________________________________________")
>>>
>>> syout= systemfunc("rm " + mdl@analdset+"_multimodel_data.nc")
>>> outfile = addfile(mdl@analdset+"_multimodel_data.nc","c")
>>> outfile->multi_model_data = multidata
>>>
>>> return(multidata)
>>> end ;;; End of function definition get_multimodel_data() ;;;
>>> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>>> _______________________________________
>>>
>>> _________________________________
>>> ___________ Output _______________
>>> _________________________________
>>>
>>>
>>> (0) ---------------Multimodel Ensemble-------------------
>>> (0) _____________________________________________________
>>>
>>>
>>> Variable: multidata
>>> Type: float
>>> Total Size: 272160000 bytes
>>> 68040000 values
>>> Number of Dimensions: 7
>>> Dimensions and sizes: [time | 25] x [lat | 280] x [lon | 360] x [season | 1] x [component | 3] x [ens | 1] x [mdl | 9]
>>> Coordinates:
>>> time: [1981..2005]
>>> lat: [-35..35]
>>> lon: [30..120]
>>> season: [ond..ond]
>>> component: [rain..tsfc]
>>> ens: [..]
>>> mdl: [cfsv2_jun_init..DePreSys_UKMO_aug_init]
>>> Number Of Attributes: 18
>>> unit : ( mm/day, hPa, degC )
>>> units : ( mm/day, hPa, degC )
>>> region : indotropic
>>> ssnend : 12
>>> ssnbegin : 10
>>> dataset : ond_skill_1981_to_2005
>>> datavar : _rain_mslp_tsfc
>>> anal_start_year : 1981
>>> anal_end_year : 2005
>>> latsouth : -35
>>> latnorth : 35
>>> lonwest : 30
>>> loneast : 120
>>> mpCenterLonF : 75
>>> novctr : 0
>>> nosclr : 3
>>> long_name : _rain_mslp_tsfc
>>> _FillValue : -9.96921e+36
>>> (0) _____________________________________________________
>>> fatal:Attempting to write variable (season) of type (string) which is not representable in the format of file (ond_skill_multimodel_data.nc)
>>> fatal:Trying using a type conversion function
>>> warning:FileWriteVarVar: Could not write coordinate variable (season) to file (ond_skill_multimodel_data), continuing anyway
>>> fatal:Attempting to write variable (component) of type (string) which is not representable in the format of file (ond_skill_multimodel_data.nc)
>>> fatal:Trying using a type conversion function
>>> warning:FileWriteVarVar: Could not write coordinate variable (component) to file (ond_skill_multimodel_data), continuing anyway
>>> fatal:Attempting to write variable (ens) of type (string) which is not representable in the format of file (ond_skill_multimodel_data.nc)
>>> fatal:Trying using a type conversion function
>>> warning:FileWriteVarVar: Could not write coordinate variable (ens) to file (ond_skill_multimodel_data), continuing anyway
>>> fatal:Attempting to write variable (mdl) of type (string) which is not representable in the format of file (ond_skill_multimodel_data.nc)
>>> fatal:Trying using a type conversion function
>>> warning:FileWriteVarVar: Could not write coordinate variable (mdl) to file (ond_skill_multimodel_data), continuing anyway
>>> warning:Execute: Error occurred at or near line 3188 in file /home/gibies/ncl/mylib/lib_gibies_read_data_module.ncl
>>>
>>>
>>>
>>> --
>>> Gibies George, CSIR-RF,
>>> Monsoon Seasonal Prediction Group,
>>> Climate and Global Modelling Division,
>>> Indian Institute of Tropical Meteorology,
>>> Dr. Homi Bhabha Road,
>>> NCL (P. O.), Pashan,
>>> Pune 411008, India.
>>>
>>> http://sites.google.com/site/gibiesge/
>>>
>>> Please Think about the environment. Save paper; Save Trees; and don't print this e-mail unless it is necessary.
>>> _______________________________________________
>>> ncl-talk mailing list
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>>
>>
>> --
>> Gibies George, CSIR-RF,
>> Monsoon Seasonal Prediction Group,
>> Climate and Global Modelling Division,
>> Indian Institute of Tropical Meteorology,
>> Dr. Homi Bhabha Road,
>> NCL (P. O.), Pashan,
>> Pune 411008, India.
>>
>> http://sites.google.com/site/gibiesge/
>>
>> Please Think about the environment. Save paper; Save Trees; and don't print this e-mail unless it is necessary.
>>
>>
>>
>> --
>> Gibies George, CSIR-RF,
>> Monsoon Seasonal Prediction Group,
>> Climate and Global Modelling Division,
>> Indian Institute of Tropical Meteorology,
>> Dr. Homi Bhabha Road,
>> NCL (P. O.), Pashan,
>> Pune 411008, India.
>>
>> http://sites.google.com/site/gibiesge/
>>
>> Please Think about the environment. Save paper; Save Trees; and don't print this e-mail unless it is necessary.
>
>
>
>
> --
>
>
> Gibies George, CSIR-RF,
> Climate and Global Modelling Division,
> Indian Institute of Tropical Meteorology,
> Dr. Homi Bhabha Road,
> NCL (P. O.), Pashan,
> Pune 411008, India.
> Office mail : gibies@tropmet.res.in
> Mobile: 9881707796 (at Pune)
>
> http://sites.google.com/site/gibiesge/
>
> Please Think about the environment. Save paper; Save Trees; and don't print this e-mail unless it is necessary.
>
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk

_______________________________________________
ncl-talk mailing list
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk
Received on Wed Aug 17 07:25:01 2011

This archive was generated by hypermail 2.1.8 : Mon Aug 22 2011 - 08:13:37 MDT