grib2io
Introduction
grib2io is a Python package that provides an interface to the NCEP GRIB2 C (g2c) library for the purpose of reading and writing WMO GRIdded Binary, Edition 2 (GRIB2) messages. A physical file can contain one or more GRIB2 messages.
GRIB2 file IO is performed directly in Python. The unpacking/packing of GRIB2 integer, coded metadata and data sections is performed by the g2c library functions via the g2clib Cython extension module. The decoding/encoding of GRIB2 metadata is translated into more descriptive, plain language metadata by providing a mapping of the integer coded metadata to the appropriate GRIB2 code tables. The NCEP GRIB2 code tables are a part of the grib2io package.
Interpolation
As of grib2io v2.4.0, spatial interpolation via NCEPLIPS-ip Fortran library is now a part of the grib2io package. The separate component package, grib2io-interp, has been deprecated. grib2io-interp provided interpolation via F2PY interface to NCEPLIBS-ip, which has become difficult since the removal of distutils from Python 3.12+.
NCEPLIBS-ip interpolation Fortran subroutines contain the BIND(C)
attribute which
provides an equivalent C-interface. grib2io now provides a Cython-based interface, iplib
,
to these Fortran subroutines via their C-interface. If NCEPLIBS-ip was built with
OpenMP support, iplib
will provide functions for getting and setting the number of
OpenMP threads.
Tutorials
The following Jupyter Notebooks are available as tutorials:
1from ._grib2io import * 2from ._grib2io import __doc__ 3from ._grib2io import _Grib2Message 4 5__all__ = ['open', 'show_config', 'interpolate', 'interpolate_to_stations', 6 'tables', 'templates', 'utils', 7 'Grib2Message', '_Grib2Message', 'Grib2GridDef'] 8 9try: 10 from . import __config__ 11 __version__ = __config__.grib2io_version 12 has_interpolation = __config__.has_interpolation 13 has_openmp_support = __config__.has_openmp_support 14 g2c_static = __config__.g2c_static 15 ip_static = __config__.ip_static 16 extra_objects = __config__.extra_objects 17except(ImportError): 18 pass 19 20from .g2clib import __version__ as __g2clib_version__ 21from .g2clib import _has_jpeg 22from .g2clib import _has_png 23from .g2clib import _has_aec 24 25has_jpeg_support = bool(_has_jpeg) 26has_png_support = bool(_has_png) 27has_aec_support = bool(_has_aec) 28 29from .tables.originating_centers import _ncep_grib2_table_version 30ncep_grib2_table_version = _ncep_grib2_table_version 31g2c_version = __g2clib_version__ 32 33def show_config(): 34 """Print grib2io build configuration information.""" 35 print(f'grib2io version {__version__} Configuration:') 36 print(f'') 37 print(f'NCEPLIBS-g2c library version: {__g2clib_version__}') 38 print(f'\tStatic library: {g2c_static}') 39 print(f'\tJPEG compression support: {has_jpeg_support}') 40 print(f'\tPNG compression support: {has_png_support}') 41 print(f'\tAEC compression support: {has_aec_support}') 42 print(f'') 43 print(f'NCEPLIPS-ip support: {has_interpolation}') 44 print(f'\tStatic library: {ip_static}') 45 print(f'\tOpenMP support: {has_openmp_support}') 46 print(f'') 47 print(f'Static libs:') 48 for lib in extra_objects: 49 print(f'\t{lib}') 50 print(f'') 51 print(f'NCEP GRIB2 Table Version: {_ncep_grib2_table_version}')
76class open(): 77 """ 78 GRIB2 File Object. 79 80 A physical file can contain one or more GRIB2 messages. When instantiated, 81 class `grib2io.open`, the file named `filename` is opened for reading (`mode 82 = 'r'`) and is automatically indexed. The indexing procedure reads some of 83 the GRIB2 metadata for all GRIB2 Messages. A GRIB2 Message may contain 84 submessages whereby Section 2-7 can be repeated. grib2io accommodates for 85 this by flattening any GRIB2 submessages into multiple individual messages. 86 87 It is important to note that GRIB2 files from some Meteorological agencies 88 contain other data than GRIB2 messages. GRIB2 files from ECMWF can contain 89 GRIB1 and GRIB2 messages. grib2io checks for these and safely ignores them. 90 91 Attributes 92 ---------- 93 closed : bool 94 `True` is file handle is close; `False` otherwise. 95 current_message : int 96 Current position of the file in units of GRIB2 Messages. 97 levels : tuple 98 Tuple containing a unique list of wgrib2-formatted level/layer strings. 99 messages : int 100 Count of GRIB2 Messages contained in the file. 101 mode : str 102 File IO mode of opening the file. 103 name : str 104 Full path name of the GRIB2 file. 105 size : int 106 Size of the file in units of bytes. 107 variables : tuple 108 Tuple containing a unique list of variable short names (i.e. GRIB2 109 abbreviation names). 110 """ 111 112 __slots__ = ('_fileid', '_filehandle', '_hasindex', '_index', '_nodata', 113 '_pos', 'closed', 'current_message', 'messages', 'mode', 114 'name', 'size') 115 116 def __init__(self, filename: str, mode: Literal["r", "w", "x"] = "r", **kwargs): 117 """ 118 Initialize GRIB2 File object instance. 119 120 Parameters 121 ---------- 122 filename 123 File name containing GRIB2 messages. 124 mode: default="r" 125 File access mode where "r" opens the files for reading only; "w" 126 opens the file for overwriting and "x" for writing to a new file. 127 """ 128 129 # Manage keywords 130 if "_xarray_backend" not in kwargs: 131 kwargs["_xarray_backend"] = False 132 self._nodata = False 133 else: 134 self._nodata = kwargs["_xarray_backend"] 135 136 # All write modes are read/write. 137 # All modes are binary. 138 if mode in ("a", "x", "w"): 139 mode += "+" 140 mode = mode + "b" 141 142 # Some GRIB2 files are gzipped, so check for that here, but 143 # raise error when using xarray backend. 144 if 'r' in mode: 145 self._filehandle = builtins.open(filename, mode=mode) 146 # Gzip files contain a 2-byte header b'\x1f\x8b'. 147 if self._filehandle.read(2) == b'\x1f\x8b': 148 self._filehandle.close() 149 if kwargs["_xarray_backend"]: 150 raise RuntimeError('Gzip GRIB2 files are not supported by the Xarray backend.') 151 import gzip 152 self._filehandle = gzip.open(filename, mode=mode) 153 else: 154 self._filehandle = builtins.open(filename, mode=mode) 155 else: 156 self._filehandle = builtins.open(filename, mode=mode) 157 158 self.name = os.path.abspath(filename) 159 fstat = os.stat(self.name) 160 self._hasindex = False 161 self._index = {} 162 self.mode = mode 163 self.messages = 0 164 self.current_message = 0 165 self.size = fstat.st_size 166 self.closed = self._filehandle.closed 167 self._fileid = hashlib.sha1((self.name+str(fstat.st_ino)+ 168 str(self.size)).encode('ASCII')).hexdigest() 169 if 'r' in self.mode: 170 self._build_index() 171 # FIX: Cannot perform reads on mode='a' 172 #if 'a' in self.mode and self.size > 0: self._build_index() 173 174 175 def __delete__(self, instance): 176 self.close() 177 del self._index 178 179 180 def __enter__(self): 181 return self 182 183 184 def __exit__(self, atype, value, traceback): 185 self.close() 186 187 188 def __iter__(self): 189 yield from self._index['msg'] 190 191 192 def __len__(self): 193 return self.messages 194 195 196 def __repr__(self): 197 strings = [] 198 for k in self.__slots__: 199 if k.startswith('_'): continue 200 strings.append('%s = %s\n'%(k,eval('self.'+k))) 201 return ''.join(strings) 202 203 204 def __getitem__(self, key): 205 if isinstance(key,int): 206 if abs(key) >= len(self._index['msg']): 207 raise IndexError("index out of range") 208 else: 209 return self._index['msg'][key] 210 elif isinstance(key,str): 211 return self.select(shortName=key) 212 elif isinstance(key,slice): 213 return self._index['msg'][key] 214 else: 215 raise KeyError('Key must be an integer, slice, or GRIB2 variable shortName.') 216 217 218 def _build_index(self): 219 """Perform indexing of GRIB2 Messages.""" 220 # Initialize index dictionary 221 if not self._hasindex: 222 self._index['sectionOffset'] = [] 223 self._index['sectionSize'] = [] 224 self._index['msgSize'] = [] 225 self._index['msgNumber'] = [] 226 self._index['msg'] = [] 227 self._index['isSubmessage'] = [] 228 self._hasindex = True 229 230 # Iterate 231 while True: 232 try: 233 # Initialize 234 bmapflag = None 235 pos = self._filehandle.tell() 236 section2 = b'' 237 trailer = b'' 238 _secpos = dict.fromkeys(range(8)) 239 _secsize = dict.fromkeys(range(8)) 240 _isSubmessage = False 241 242 # Ignore headers (usually text) that are not part of the GRIB2 243 # file. For example, NAVGEM files have a http header at the 244 # beginning that needs to be ignored. 245 246 # Read a byte at a time until "GRIB" is found. Using 247 # "wgrib2" on a NAVGEM file, the header was 421 bytes and 248 # decided to go to 2048 bytes to be safe. For normal GRIB2 249 # files this should be quick and break out of the first 250 # loop when "test_offset" is 0. 251 for test_offset in range(2048): 252 self._filehandle.seek(pos + test_offset) 253 header = struct.unpack(">I", self._filehandle.read(4))[0] 254 if header.to_bytes(4, "big") == b"GRIB": 255 pos = pos + test_offset 256 break 257 else: 258 # NOTE: Coming here means that no "GRIB" message identifier 259 # was found in the previous 2048 bytes. So here we continue 260 # the while True loop. 261 continue 262 263 # Read the rest of Section 0 using struct. 264 _secpos[0] = self._filehandle.tell()-4 265 _secsize[0] = 16 266 secmsg = self._filehandle.read(12) 267 section0 = np.concatenate(([header],list(struct.unpack('>HBBQ',secmsg))),dtype=np.int64) 268 269 # Make sure message is GRIB2. 270 if section0[3] != 2: 271 # Check for GRIB1 and ignore. 272 if secmsg[3] == 1: 273 warnings.warn("GRIB version 1 message detected. Ignoring...") 274 grib1_size = int.from_bytes(secmsg[:3],"big") 275 self._filehandle.seek(self._filehandle.tell()+grib1_size-16) 276 continue 277 else: 278 raise ValueError("Bad GRIB version number.") 279 280 # Read and unpack sections 1 through 8 which all follow a 281 # pattern of section size, number, and content. 282 while 1: 283 # Read first 5 bytes of the section which contains the size 284 # of the section (4 bytes) and the section number (1 byte). 285 secmsg = self._filehandle.read(5) 286 secsize, secnum = struct.unpack('>iB',secmsg) 287 288 # Record the offset of the section number and "append" the 289 # rest of the section to secmsg. 290 _secpos[secnum] = self._filehandle.tell()-5 291 _secsize[secnum] = secsize 292 if secnum in {1,3,4,5}: 293 secmsg += self._filehandle.read(secsize-5) 294 grbpos = 0 295 296 # Unpack section 297 if secnum == 1: 298 # Unpack Section 1 299 section1, grbpos = g2clib.unpack1(secmsg) 300 elif secnum == 2: 301 # Unpack Section 2 302 section2 = self._filehandle.read(secsize-5) 303 elif secnum == 3: 304 # Unpack Section 3 305 gds, gdt, deflist, grbpos = g2clib.unpack3(secmsg) 306 gds = gds.tolist() 307 gdt = gdt.tolist() 308 section3 = np.concatenate((gds,gdt)) 309 section3 = np.where(section3==4294967295,-1,section3) 310 elif secnum == 4: 311 # Unpack Section 4 312 pdtnum, pdt, coordlist, numcoord, grbpos = g2clib.unpack4(secmsg) 313 pdt = pdt.tolist() 314 section4 = np.concatenate((np.array((numcoord,pdtnum)),pdt)) 315 elif secnum == 5: 316 # Unpack Section 5 317 drtn, drt, npts, self._pos = g2clib.unpack5(secmsg) 318 section5 = np.concatenate((np.array((npts,drtn)),drt)) 319 section5 = np.where(section5==4294967295,-1,section5) 320 elif secnum == 6: 321 # Unpack Section 6 - Just the bitmap flag 322 bmapflag = struct.unpack('>B',self._filehandle.read(1))[0] 323 if bmapflag == 0: 324 bmappos = self._filehandle.tell()-6 325 elif bmapflag == 254: 326 # Do this to keep the previous position value 327 pass 328 else: 329 bmappos = None 330 self._filehandle.seek(self._filehandle.tell()+secsize-6) 331 elif secnum == 7: 332 # Do not unpack section 7 here, but move the file pointer 333 # to after section 7. 334 self._filehandle.seek(self._filehandle.tell()+secsize-5) 335 336 # Update the file index. 337 self.messages += 1 338 self._index['sectionOffset'].append(copy.deepcopy(_secpos)) 339 self._index['sectionSize'].append(copy.deepcopy(_secsize)) 340 self._index['msgSize'].append(section0[-1]) 341 self._index['msgNumber'].append(self.messages) 342 self._index['isSubmessage'].append(_isSubmessage) 343 344 # Create Grib2Message with data. 345 msg = Grib2Message(section0,section1,section2,section3,section4,section5,bmapflag) 346 msg._msgnum = self.messages-1 347 msg._deflist = deflist 348 msg._coordlist = coordlist 349 if not self._nodata: 350 msg._ondiskarray = Grib2MessageOnDiskArray((msg.ny,msg.nx), 2, 351 TYPE_OF_VALUES_DTYPE[msg.typeOfValues], 352 self._filehandle, 353 msg, pos, _secpos[6], _secpos[7]) 354 self._index['msg'].append(msg) 355 356 # If here, then we have moved through GRIB2 section 1-7. 357 # Now we need to check the next 4 bytes after section 7. 358 trailer = struct.unpack('>i',self._filehandle.read(4))[0] 359 360 # If we reach the GRIB2 trailer string ('7777'), then we 361 # can break begin processing the next GRIB2 message. If 362 # not, then we continue within the same iteration to 363 # process a GRIB2 submessage. 364 if trailer.to_bytes(4, "big") == b'7777': 365 break 366 else: 367 # If here, trailer should be the size of the first 368 # section of the next submessage, then the next byte 369 # is the section number. Check this value. 370 nextsec = struct.unpack('>B',self._filehandle.read(1))[0] 371 if nextsec not in {2,3,4}: 372 raise ValueError("Bad GRIB2 message structure.") 373 self._filehandle.seek(self._filehandle.tell()-5) 374 _isSubmessage = True 375 continue 376 else: 377 raise ValueError("Bad GRIB2 section number.") 378 379 except(struct.error): 380 if 'r' in self.mode: 381 self._filehandle.seek(0) 382 break 383 384 385 @property 386 def levels(self): 387 """Provides a unique tuple of level strings.""" 388 if self._hasindex and not self._nodata: 389 return tuple(sorted(set([msg.level for msg in self._index['msg']]))) 390 else: 391 return None 392 393 394 @property 395 def variables(self): 396 """Provides a unique tuple of variable shortName strings.""" 397 if self._hasindex and not self._nodata: 398 return tuple(sorted(set([msg.shortName for msg in self._index['msg']]))) 399 else: 400 return None 401 402 403 def close(self): 404 """Close the file handle.""" 405 if not self._filehandle.closed: 406 self.messages = 0 407 self.current_message = 0 408 self._filehandle.close() 409 self.closed = self._filehandle.closed 410 411 412 def read(self, size: Optional[int]=None): 413 """ 414 Read size amount of GRIB2 messages from the current position. 415 416 If no argument is given, then size is None and all messages are returned 417 from the current position in the file. This read method follows the 418 behavior of Python's builtin open() function, but whereas that operates 419 on units of bytes, we operate on units of GRIB2 messages. 420 421 Parameters 422 ---------- 423 size: default=None 424 The number of GRIB2 messages to read from the current position. If 425 no argument is give, the default value is None and remainder of 426 the file is read. 427 428 Returns 429 ------- 430 read 431 ``Grib2Message`` object when size = 1 or a list of Grib2Messages 432 when size > 1. 433 """ 434 if size is not None and size < 0: 435 size = None 436 if size is None or size > 1: 437 start = self.tell() 438 stop = self.messages if size is None else start+size 439 if size is None: 440 self.current_message = self.messages-1 441 else: 442 self.current_message += size 443 return self._index['msg'][slice(start,stop,1)] 444 elif size == 1: 445 self.current_message += 1 446 return self._index['msg'][self.current_message] 447 else: 448 None 449 450 451 def seek(self, pos: int): 452 """ 453 Set the position within the file in units of GRIB2 messages. 454 455 Parameters 456 ---------- 457 pos 458 The GRIB2 Message number to set the file pointer to. 459 """ 460 if self._hasindex: 461 self._filehandle.seek(self._index['sectionOffset'][0][pos]) 462 self.current_message = pos 463 464 465 def tell(self): 466 """Returns the position of the file in units of GRIB2 Messages.""" 467 return self.current_message 468 469 470 def select(self, **kwargs): 471 """Select GRIB2 messages by `Grib2Message` attributes.""" 472 # TODO: Added ability to process multiple values for each keyword (attribute) 473 idxs = [] 474 nkeys = len(kwargs.keys()) 475 for k,v in kwargs.items(): 476 for m in self._index['msg']: 477 if hasattr(m,k) and getattr(m,k) == v: idxs.append(m._msgnum) 478 idxs = np.array(idxs,dtype=np.int32) 479 return [self._index['msg'][i] for i in [ii[0] for ii in collections.Counter(idxs).most_common() if ii[1] == nkeys]] 480 481 482 def write(self, msg): 483 """ 484 Writes GRIB2 message object to file. 485 486 Parameters 487 ---------- 488 msg 489 GRIB2 message objects to write to file. 490 """ 491 if isinstance(msg,list): 492 for m in msg: 493 self.write(m) 494 return 495 496 if issubclass(msg.__class__,_Grib2Message): 497 if hasattr(msg,'_msg'): 498 self._filehandle.write(msg._msg) 499 else: 500 if msg._signature != msg._generate_signature(): 501 msg.pack() 502 self._filehandle.write(msg._msg) 503 else: 504 if hasattr(msg._data,'filehandle'): 505 msg._data.filehandle.seek(msg._data.offset) 506 self._filehandle.write(msg._data.filehandle.read(msg.section0[-1])) 507 else: 508 msg.pack() 509 self._filehandle.write(msg._msg) 510 self.flush() 511 self.size = os.path.getsize(self.name) 512 self._filehandle.seek(self.size-msg.section0[-1]) 513 self._build_index() 514 else: 515 raise TypeError("msg must be a Grib2Message object.") 516 return 517 518 519 def flush(self): 520 """Flush the file object buffer.""" 521 self._filehandle.flush() 522 523 524 def levels_by_var(self, name: str): 525 """ 526 Return a list of level strings given a variable shortName. 527 528 Parameters 529 ---------- 530 name 531 Grib2Message variable shortName 532 533 Returns 534 ------- 535 levels_by_var 536 A list of unique level strings. 537 """ 538 return list(sorted(set([msg.level for msg in self.select(shortName=name)]))) 539 540 541 def vars_by_level(self, level: str): 542 """ 543 Return a list of variable shortName strings given a level. 544 545 Parameters 546 ---------- 547 level 548 Grib2Message variable level 549 550 Returns 551 ------- 552 vars_by_level 553 A list of unique variable shortName strings. 554 """ 555 return list(sorted(set([msg.shortName for msg in self.select(level=level)])))
GRIB2 File Object.
A physical file can contain one or more GRIB2 messages. When instantiated,
class grib2io.open
, the file named filename
is opened for reading (mode
= 'r'
) and is automatically indexed. The indexing procedure reads some of
the GRIB2 metadata for all GRIB2 Messages. A GRIB2 Message may contain
submessages whereby Section 2-7 can be repeated. grib2io accommodates for
this by flattening any GRIB2 submessages into multiple individual messages.
It is important to note that GRIB2 files from some Meteorological agencies contain other data than GRIB2 messages. GRIB2 files from ECMWF can contain GRIB1 and GRIB2 messages. grib2io checks for these and safely ignores them.
Attributes
- closed (bool):
True
is file handle is close;False
otherwise. - current_message (int): Current position of the file in units of GRIB2 Messages.
- levels (tuple): Tuple containing a unique list of wgrib2-formatted level/layer strings.
- messages (int): Count of GRIB2 Messages contained in the file.
- mode (str): File IO mode of opening the file.
- name (str): Full path name of the GRIB2 file.
- size (int): Size of the file in units of bytes.
- variables (tuple): Tuple containing a unique list of variable short names (i.e. GRIB2 abbreviation names).
116 def __init__(self, filename: str, mode: Literal["r", "w", "x"] = "r", **kwargs): 117 """ 118 Initialize GRIB2 File object instance. 119 120 Parameters 121 ---------- 122 filename 123 File name containing GRIB2 messages. 124 mode: default="r" 125 File access mode where "r" opens the files for reading only; "w" 126 opens the file for overwriting and "x" for writing to a new file. 127 """ 128 129 # Manage keywords 130 if "_xarray_backend" not in kwargs: 131 kwargs["_xarray_backend"] = False 132 self._nodata = False 133 else: 134 self._nodata = kwargs["_xarray_backend"] 135 136 # All write modes are read/write. 137 # All modes are binary. 138 if mode in ("a", "x", "w"): 139 mode += "+" 140 mode = mode + "b" 141 142 # Some GRIB2 files are gzipped, so check for that here, but 143 # raise error when using xarray backend. 144 if 'r' in mode: 145 self._filehandle = builtins.open(filename, mode=mode) 146 # Gzip files contain a 2-byte header b'\x1f\x8b'. 147 if self._filehandle.read(2) == b'\x1f\x8b': 148 self._filehandle.close() 149 if kwargs["_xarray_backend"]: 150 raise RuntimeError('Gzip GRIB2 files are not supported by the Xarray backend.') 151 import gzip 152 self._filehandle = gzip.open(filename, mode=mode) 153 else: 154 self._filehandle = builtins.open(filename, mode=mode) 155 else: 156 self._filehandle = builtins.open(filename, mode=mode) 157 158 self.name = os.path.abspath(filename) 159 fstat = os.stat(self.name) 160 self._hasindex = False 161 self._index = {} 162 self.mode = mode 163 self.messages = 0 164 self.current_message = 0 165 self.size = fstat.st_size 166 self.closed = self._filehandle.closed 167 self._fileid = hashlib.sha1((self.name+str(fstat.st_ino)+ 168 str(self.size)).encode('ASCII')).hexdigest() 169 if 'r' in self.mode: 170 self._build_index() 171 # FIX: Cannot perform reads on mode='a' 172 #if 'a' in self.mode and self.size > 0: self._build_index()
Initialize GRIB2 File object instance.
Parameters
- filename: File name containing GRIB2 messages.
- mode (default="r"): File access mode where "r" opens the files for reading only; "w" opens the file for overwriting and "x" for writing to a new file.
385 @property 386 def levels(self): 387 """Provides a unique tuple of level strings.""" 388 if self._hasindex and not self._nodata: 389 return tuple(sorted(set([msg.level for msg in self._index['msg']]))) 390 else: 391 return None
Provides a unique tuple of level strings.
394 @property 395 def variables(self): 396 """Provides a unique tuple of variable shortName strings.""" 397 if self._hasindex and not self._nodata: 398 return tuple(sorted(set([msg.shortName for msg in self._index['msg']]))) 399 else: 400 return None
Provides a unique tuple of variable shortName strings.
403 def close(self): 404 """Close the file handle.""" 405 if not self._filehandle.closed: 406 self.messages = 0 407 self.current_message = 0 408 self._filehandle.close() 409 self.closed = self._filehandle.closed
Close the file handle.
412 def read(self, size: Optional[int]=None): 413 """ 414 Read size amount of GRIB2 messages from the current position. 415 416 If no argument is given, then size is None and all messages are returned 417 from the current position in the file. This read method follows the 418 behavior of Python's builtin open() function, but whereas that operates 419 on units of bytes, we operate on units of GRIB2 messages. 420 421 Parameters 422 ---------- 423 size: default=None 424 The number of GRIB2 messages to read from the current position. If 425 no argument is give, the default value is None and remainder of 426 the file is read. 427 428 Returns 429 ------- 430 read 431 ``Grib2Message`` object when size = 1 or a list of Grib2Messages 432 when size > 1. 433 """ 434 if size is not None and size < 0: 435 size = None 436 if size is None or size > 1: 437 start = self.tell() 438 stop = self.messages if size is None else start+size 439 if size is None: 440 self.current_message = self.messages-1 441 else: 442 self.current_message += size 443 return self._index['msg'][slice(start,stop,1)] 444 elif size == 1: 445 self.current_message += 1 446 return self._index['msg'][self.current_message] 447 else: 448 None
Read size amount of GRIB2 messages from the current position.
If no argument is given, then size is None and all messages are returned from the current position in the file. This read method follows the behavior of Python's builtin open() function, but whereas that operates on units of bytes, we operate on units of GRIB2 messages.
Parameters
- size (default=None): The number of GRIB2 messages to read from the current position. If no argument is give, the default value is None and remainder of the file is read.
Returns
- read:
Grib2Message
object when size = 1 or a list of Grib2Messages when size > 1.
451 def seek(self, pos: int): 452 """ 453 Set the position within the file in units of GRIB2 messages. 454 455 Parameters 456 ---------- 457 pos 458 The GRIB2 Message number to set the file pointer to. 459 """ 460 if self._hasindex: 461 self._filehandle.seek(self._index['sectionOffset'][0][pos]) 462 self.current_message = pos
Set the position within the file in units of GRIB2 messages.
Parameters
- pos: The GRIB2 Message number to set the file pointer to.
465 def tell(self): 466 """Returns the position of the file in units of GRIB2 Messages.""" 467 return self.current_message
Returns the position of the file in units of GRIB2 Messages.
470 def select(self, **kwargs): 471 """Select GRIB2 messages by `Grib2Message` attributes.""" 472 # TODO: Added ability to process multiple values for each keyword (attribute) 473 idxs = [] 474 nkeys = len(kwargs.keys()) 475 for k,v in kwargs.items(): 476 for m in self._index['msg']: 477 if hasattr(m,k) and getattr(m,k) == v: idxs.append(m._msgnum) 478 idxs = np.array(idxs,dtype=np.int32) 479 return [self._index['msg'][i] for i in [ii[0] for ii in collections.Counter(idxs).most_common() if ii[1] == nkeys]]
Select GRIB2 messages by Grib2Message
attributes.
482 def write(self, msg): 483 """ 484 Writes GRIB2 message object to file. 485 486 Parameters 487 ---------- 488 msg 489 GRIB2 message objects to write to file. 490 """ 491 if isinstance(msg,list): 492 for m in msg: 493 self.write(m) 494 return 495 496 if issubclass(msg.__class__,_Grib2Message): 497 if hasattr(msg,'_msg'): 498 self._filehandle.write(msg._msg) 499 else: 500 if msg._signature != msg._generate_signature(): 501 msg.pack() 502 self._filehandle.write(msg._msg) 503 else: 504 if hasattr(msg._data,'filehandle'): 505 msg._data.filehandle.seek(msg._data.offset) 506 self._filehandle.write(msg._data.filehandle.read(msg.section0[-1])) 507 else: 508 msg.pack() 509 self._filehandle.write(msg._msg) 510 self.flush() 511 self.size = os.path.getsize(self.name) 512 self._filehandle.seek(self.size-msg.section0[-1]) 513 self._build_index() 514 else: 515 raise TypeError("msg must be a Grib2Message object.") 516 return
Writes GRIB2 message object to file.
Parameters
- msg: GRIB2 message objects to write to file.
524 def levels_by_var(self, name: str): 525 """ 526 Return a list of level strings given a variable shortName. 527 528 Parameters 529 ---------- 530 name 531 Grib2Message variable shortName 532 533 Returns 534 ------- 535 levels_by_var 536 A list of unique level strings. 537 """ 538 return list(sorted(set([msg.level for msg in self.select(shortName=name)])))
Return a list of level strings given a variable shortName.
Parameters
- name: Grib2Message variable shortName
Returns
- levels_by_var: A list of unique level strings.
541 def vars_by_level(self, level: str): 542 """ 543 Return a list of variable shortName strings given a level. 544 545 Parameters 546 ---------- 547 level 548 Grib2Message variable level 549 550 Returns 551 ------- 552 vars_by_level 553 A list of unique variable shortName strings. 554 """ 555 return list(sorted(set([msg.shortName for msg in self.select(level=level)])))
Return a list of variable shortName strings given a level.
Parameters
- level: Grib2Message variable level
Returns
- vars_by_level: A list of unique variable shortName strings.
34def show_config(): 35 """Print grib2io build configuration information.""" 36 print(f'grib2io version {__version__} Configuration:') 37 print(f'') 38 print(f'NCEPLIBS-g2c library version: {__g2clib_version__}') 39 print(f'\tStatic library: {g2c_static}') 40 print(f'\tJPEG compression support: {has_jpeg_support}') 41 print(f'\tPNG compression support: {has_png_support}') 42 print(f'\tAEC compression support: {has_aec_support}') 43 print(f'') 44 print(f'NCEPLIPS-ip support: {has_interpolation}') 45 print(f'\tStatic library: {ip_static}') 46 print(f'\tOpenMP support: {has_openmp_support}') 47 print(f'') 48 print(f'Static libs:') 49 for lib in extra_objects: 50 print(f'\t{lib}') 51 print(f'') 52 print(f'NCEP GRIB2 Table Version: {_ncep_grib2_table_version}')
Print grib2io build configuration information.
1670def interpolate(a, 1671 method: Union[int, str], 1672 grid_def_in, 1673 grid_def_out, 1674 method_options=None, 1675 num_threads=1, 1676): 1677 """ 1678 This is the module-level interpolation function. 1679 1680 This interfaces with the 4-byte (32-bit float) [NCEPLIBS-ip library](https://github.com/NOAA-EMC/NCEPLIBS-ip) 1681 through grib2io's internal iplib Cython extension module. 1682 1683 Parameters 1684 ---------- 1685 a : numpy.ndarray or tuple 1686 Input data. If `a` is a `numpy.ndarray`, scalar interpolation will be 1687 performed. If `a` is a `tuple`, then vector interpolation will be 1688 performed with the assumption that u = a[0] and v = a[1] and are both 1689 `numpy.ndarray`. 1690 1691 These data are expected to be in 2-dimensional form with shape (ny, nx) 1692 or 3-dimensional (:, ny, nx) where the 1st dimension represents another 1693 spatial, temporal, or classification (i.e. ensemble members) dimension. 1694 The function will properly flatten the (ny,nx) dimensions into (nx * ny) 1695 acceptable for input into the interpolation subroutines. If needed, these 1696 data will be converted to `np.float32`. 1697 method 1698 Interpolate method to use. This can either be an integer or string using 1699 the following mapping: 1700 1701 | Interpolate Scheme | Integer Value | 1702 | :---: | :---: | 1703 | 'bilinear' | 0 | 1704 | 'bicubic' | 1 | 1705 | 'neighbor' | 2 | 1706 | 'budget' | 3 | 1707 | 'spectral' | 4 | 1708 | 'neighbor-budget' | 6 | 1709 1710 grid_def_in : grib2io.Grib2GridDef 1711 Grib2GridDef object for the input grid. 1712 grid_def_out : grib2io.Grib2GridDef 1713 Grib2GridDef object for the output grid or station points. 1714 method_options : list of ints, optional 1715 Interpolation options. See the NCEPLIBS-ip documentation for 1716 more information on how these are used. 1717 num_threads : int, optional 1718 Number of OpenMP threads to use for interpolation. The default 1719 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1720 was not built with OpenMP, then this keyword argument and value 1721 will have no impact. 1722 1723 Returns 1724 ------- 1725 interpolate 1726 Returns a `numpy.ndarray` of dtype `np.float32` when scalar interpolation 1727 is performed or a `tuple` of `numpy.ndarray`s when vector interpolation is 1728 performed with the assumptions that 0-index is the interpolated u and 1729 1-index is the interpolated v. 1730 """ 1731 1732 from . import iplib 1733 1734 prev_num_threads = 1 1735 try: 1736 prev_num_threads = iplib.get_openmp_threads() 1737 iplib.set_openmp_threads(num_threads) 1738 except(AttributeError): 1739 pass 1740 1741 if isinstance(method,int) and method not in _interp_schemes.values(): 1742 raise ValueError('Invalid interpolation method.') 1743 elif isinstance(method,str): 1744 if method in _interp_schemes.keys(): 1745 method = _interp_schemes[method] 1746 else: 1747 raise ValueError('Invalid interpolation method.') 1748 1749 if method_options is None: 1750 method_options = np.zeros((20),dtype=np.int32) 1751 if method in {3,6}: 1752 method_options[0:2] = -1 1753 1754 mi = grid_def_in.npoints 1755 mo = grid_def_out.npoints 1756 1757 # Adjust shape of input array(s) 1758 a, newshp = _adjust_array_shape_for_interp(a, grid_def_in, grid_def_out) 1759 1760 # Call interpolation subroutines according to type of a. 1761 if isinstance(a,np.ndarray): 1762 # Scalar 1763 km = a.shape[0] 1764 if np.any(np.isnan(a)): 1765 ibi = np.ones((km), dtype=np.int32) 1766 li = np.where(np.isnan(a),0,1).astype(np.uint8) 1767 else: 1768 ibi = np.zeros((km), dtype=np.int32) 1769 li = np.zeros(a.shape,dtype=np.uint8) 1770 no, rlat, rlon, ibo, lo, go, iret = iplib.interpolate_scalar( 1771 method, 1772 method_options, 1773 grid_def_in.gdtn, 1774 np.array(grid_def_in.gdt, dtype=np.int32), 1775 grid_def_out.gdtn, 1776 np.array(grid_def_out.gdt, dtype=np.int32), 1777 mi, 1778 mo, 1779 km, 1780 ibi, 1781 li, 1782 a, 1783 ) 1784 out = np.where(lo==0,np.nan,go).reshape(newshp) 1785 elif isinstance(a,tuple): 1786 # Vector 1787 km = a[0].shape[0] 1788 if np.any(np.isnan(a)): 1789 ibi = np.ones((km), dtype=np.int32) 1790 li = np.where(np.isnan(a),0,1).astype(np.uint8) 1791 else: 1792 ibi = np.zeros((km), dtype=np.int32) 1793 li = np.zeros(a[0].shape,dtype=np.uint8) 1794 no, rlat, rlon, crot, srot, ibo, lo, uo, vo, iret = iplib.interpolate_vector( 1795 method, 1796 method_options, 1797 grid_def_in.gdtn, 1798 np.array(grid_def_in.gdt, dtype=np.int32), 1799 grid_def_out.gdtn, 1800 np.array(grid_def_out.gdt, dtype=np.int32), 1801 mi, 1802 mo, 1803 km, 1804 ibi, 1805 li, 1806 a[0], 1807 a[1], 1808 ) 1809 uo = np.where(lo==0,np.nan,uo).reshape(newshp) 1810 vo = np.where(lo==0,np.nan,vo).reshape(newshp) 1811 out = (uo,vo) 1812 1813 try: 1814 iplib.set_openmp_threads(prev_num_threads) 1815 except(AttributeError): 1816 pass 1817 1818 return out
This is the module-level interpolation function.
This interfaces with the 4-byte (32-bit float) NCEPLIBS-ip library through grib2io's internal iplib Cython extension module.
Parameters
a (numpy.ndarray or tuple): Input data. If
a
is anumpy.ndarray
, scalar interpolation will be performed. Ifa
is atuple
, then vector interpolation will be performed with the assumption that u = a[0] and v = a[1] and are bothnumpy.ndarray
.These data are expected to be in 2-dimensional form with shape (ny, nx) or 3-dimensional (:, ny, nx) where the 1st dimension represents another spatial, temporal, or classification (i.e. ensemble members) dimension. The function will properly flatten the (ny,nx) dimensions into (nx * ny) acceptable for input into the interpolation subroutines. If needed, these data will be converted to
np.float32
.- method: Interpolate method to use. This can either be an integer or string using the following mapping:
Interpolate Scheme | Integer Value |
---|---|
'bilinear' | 0 |
'bicubic' | 1 |
'neighbor' | 2 |
'budget' | 3 |
'spectral' | 4 |
'neighbor-budget' | 6 |
- grid_def_in (grib2io.Grib2GridDef): Grib2GridDef object for the input grid.
- grid_def_out (grib2io.Grib2GridDef): Grib2GridDef object for the output grid or station points.
- method_options (list of ints, optional): Interpolation options. See the NCEPLIBS-ip documentation for more information on how these are used.
- num_threads (int, optional): Number of OpenMP threads to use for interpolation. The default value is 1. If NCEPLIBS-ip and grib2io's iplib extension module was not built with OpenMP, then this keyword argument and value will have no impact.
Returns
- interpolate: Returns a
numpy.ndarray
of dtypenp.float32
when scalar interpolation is performed or atuple
ofnumpy.ndarray
s when vector interpolation is performed with the assumptions that 0-index is the interpolated u and 1-index is the interpolated v.
1821def interpolate_to_stations( 1822 a, 1823 method: Union[int, str], 1824 grid_def_in, 1825 lats, 1826 lons, 1827 method_options=None, 1828 num_threads=1 1829): 1830 """ 1831 Module-level interpolation function for interpolation to stations. 1832 1833 Interfaces with the 4-byte (32-bit float) [NCEPLIBS-ip library](https://github.com/NOAA-EMC/NCEPLIBS-ip) 1834 via grib2io's iplib Cython exntension module. It supports scalar and 1835 vector interpolation according to the type of object a. 1836 1837 Parameters 1838 ---------- 1839 a : numpy.ndarray or tuple 1840 Input data. If `a` is a `numpy.ndarray`, scalar interpolation will be 1841 performed. If `a` is a `tuple`, then vector interpolation will be 1842 performed with the assumption that u = a[0] and v = a[1] and are both 1843 `numpy.ndarray`. 1844 1845 These data are expected to be in 2-dimensional form with shape (ny, nx) 1846 or 3-dimensional (:, ny, nx) where the 1st dimension represents another 1847 spatial, temporal, or classification (i.e. ensemble members) dimension. 1848 The function will properly flatten the (ny,nx) dimensions into (nx * ny) 1849 acceptable for input into the interpolation subroutines. If needed, these 1850 data will be converted to `np.float32`. 1851 method 1852 Interpolate method to use. This can either be an integer or string using 1853 the following mapping: 1854 1855 | Interpolate Scheme | Integer Value | 1856 | :---: | :---: | 1857 | 'bilinear' | 0 | 1858 | 'bicubic' | 1 | 1859 | 'neighbor' | 2 | 1860 | 'budget' | 3 | 1861 | 'spectral' | 4 | 1862 | 'neighbor-budget' | 6 | 1863 1864 grid_def_in : grib2io.Grib2GridDef 1865 Grib2GridDef object for the input grid. 1866 lats : numpy.ndarray or list 1867 Latitudes for station points 1868 lons : numpy.ndarray or list 1869 Longitudes for station points 1870 method_options : list of ints, optional 1871 Interpolation options. See the NCEPLIBS-ip documentation for 1872 more information on how these are used. 1873 num_threads : int, optional 1874 Number of OpenMP threads to use for interpolation. The default 1875 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1876 was not built with OpenMP, then this keyword argument and value 1877 will have no impact. 1878 1879 Returns 1880 ------- 1881 interpolate_to_stations 1882 Returns a `numpy.ndarray` of dtype `np.float32` when scalar 1883 interpolation is performed or a `tuple` of `numpy.ndarray`s 1884 when vector interpolation is performed with the assumptions 1885 that 0-index is the interpolated u and 1-index is the 1886 interpolated v. 1887 """ 1888 from . import iplib 1889 1890 prev_num_threads = 1 1891 try: 1892 prev_num_threads = iplib.get_openmp_threads() 1893 iplib.set_openmp_threads(num_threads) 1894 except(AttributeError): 1895 pass 1896 1897 if isinstance(method,int) and method not in _interp_schemes.values(): 1898 raise ValueError('Invalid interpolation method.') 1899 elif isinstance(method,str): 1900 if method in _interp_schemes.keys(): 1901 method = _interp_schemes[method] 1902 else: 1903 raise ValueError('Invalid interpolation method.') 1904 1905 if method_options is None: 1906 method_options = np.zeros((20),dtype=np.int32) 1907 if method in {3,6}: 1908 method_options[0:2] = -1 1909 1910 # Check lats and lons 1911 if isinstance(lats,list): 1912 nlats = len(lats) 1913 elif isinstance(lats,np.ndarray) and len(lats.shape) == 1: 1914 nlats = lats.shape[0] 1915 else: 1916 raise ValueError('Station latitudes must be a list or 1-D NumPy array.') 1917 if isinstance(lons,list): 1918 nlons = len(lons) 1919 elif isinstance(lons,np.ndarray) and len(lons.shape) == 1: 1920 nlons = lons.shape[0] 1921 else: 1922 raise ValueError('Station longitudes must be a list or 1-D NumPy array.') 1923 if nlats != nlons: 1924 raise ValueError('Station lats and lons must be same size.') 1925 1926 mi = grid_def_in.npoints 1927 mo = nlats 1928 1929 # Adjust shape of input array(s) 1930 a, newshp = _adjust_array_shape_for_interp_stations(a, grid_def_in, mo) 1931 1932 # Use gdtn = -1 for stations and an empty template array 1933 gdtn = -1 1934 gdt = np.zeros((200),dtype=np.int32) 1935 1936 # Call interpolation subroutines according to type of a. 1937 if isinstance(a,np.ndarray): 1938 # Scalar 1939 km = a.shape[0] 1940 ibi = np.zeros((km), dtype=np.int32) 1941 li = np.zeros(a.shape,dtype=np.uint8) 1942 no, rlat, rlon, ibo, lo, go, iret = iplib.interpolate_scalar( 1943 method, 1944 method_options, 1945 grid_def_in.gdtn, 1946 np.array(grid_def_in.gdt, dtype=np.int32), 1947 gdtn, 1948 gdt, 1949 mi, 1950 mo, 1951 km, 1952 ibi, 1953 li, 1954 a, 1955 lats=np.array(lats, dtype=np.float32), 1956 lons=np.array(lons, dtype=np.float32), 1957 ) 1958 out = go.reshape(newshp) 1959 1960 elif isinstance(a,tuple): 1961 # Vector 1962 km = a[0].shape[0] 1963 ibi = np.zeros((km), dtype=np.int32) 1964 li = np.zeros(a[0].shape,dtype=np.uint8) 1965 no, rlat, rlon, crot, srot, ibo, lo, uo, vo, iret = iplib.interpolate_vector( 1966 method, 1967 method_options, 1968 grid_def_in.gdtn, 1969 np.array(grid_def_in.gdt, dtype=np.int32), 1970 gdtn, 1971 gdt, 1972 mi, 1973 mo, 1974 km, 1975 ibi, 1976 li, 1977 a[0], 1978 a[1], 1979 lats=np.array(lats, dtype=np.float32), 1980 lons=np.array(lons, dtype=np.float32), 1981 ) 1982 out = (uo.reshape(newshp),vo.reshape(newshp)) 1983 1984 try: 1985 iplib.set_openmp_threads(prev_num_threads) 1986 except(AttributeError): 1987 pass 1988 1989 return out
Module-level interpolation function for interpolation to stations.
Interfaces with the 4-byte (32-bit float) NCEPLIBS-ip library via grib2io's iplib Cython exntension module. It supports scalar and vector interpolation according to the type of object a.
Parameters
a (numpy.ndarray or tuple): Input data. If
a
is anumpy.ndarray
, scalar interpolation will be performed. Ifa
is atuple
, then vector interpolation will be performed with the assumption that u = a[0] and v = a[1] and are bothnumpy.ndarray
.These data are expected to be in 2-dimensional form with shape (ny, nx) or 3-dimensional (:, ny, nx) where the 1st dimension represents another spatial, temporal, or classification (i.e. ensemble members) dimension. The function will properly flatten the (ny,nx) dimensions into (nx * ny) acceptable for input into the interpolation subroutines. If needed, these data will be converted to
np.float32
.- method: Interpolate method to use. This can either be an integer or string using the following mapping:
Interpolate Scheme | Integer Value |
---|---|
'bilinear' | 0 |
'bicubic' | 1 |
'neighbor' | 2 |
'budget' | 3 |
'spectral' | 4 |
'neighbor-budget' | 6 |
- grid_def_in (grib2io.Grib2GridDef): Grib2GridDef object for the input grid.
- lats (numpy.ndarray or list): Latitudes for station points
- lons (numpy.ndarray or list): Longitudes for station points
- method_options (list of ints, optional): Interpolation options. See the NCEPLIBS-ip documentation for more information on how these are used.
- num_threads (int, optional): Number of OpenMP threads to use for interpolation. The default value is 1. If NCEPLIBS-ip and grib2io's iplib extension module was not built with OpenMP, then this keyword argument and value will have no impact.
Returns
- interpolate_to_stations: Returns a
numpy.ndarray
of dtypenp.float32
when scalar interpolation is performed or atuple
ofnumpy.ndarray
s when vector interpolation is performed with the assumptions that 0-index is the interpolated u and 1-index is the interpolated v.
558class Grib2Message: 559 """ 560 Creation class for a GRIB2 message. 561 562 This class returns a dynamically-created Grib2Message object that 563 inherits from `_Grib2Message` and grid, product, data representation 564 template classes according to the template numbers for the respective 565 sections. If `section3`, `section4`, or `section5` are omitted, then 566 the appropriate keyword arguments for the template number `gdtn=`, 567 `pdtn=`, or `drtn=` must be provided. 568 569 Parameters 570 ---------- 571 section0 572 GRIB2 section 0 array. 573 section1 574 GRIB2 section 1 array. 575 section2 576 Local Use section data. 577 section3 578 GRIB2 section 3 array. 579 section4 580 GRIB2 section 4 array. 581 section5 582 GRIB2 section 5 array. 583 584 Returns 585 ------- 586 Msg 587 A dynamically-create Grib2Message object that inherits from 588 _Grib2Message, a grid definition template class, product 589 definition template class, and a data representation template 590 class. 591 """ 592 def __new__(self, section0: NDArray = np.array([struct.unpack('>I',b'GRIB')[0],0,0,2,0]), 593 section1: NDArray = np.zeros((13),dtype=np.int64), 594 section2: Optional[bytes] = None, 595 section3: Optional[NDArray] = None, 596 section4: Optional[NDArray] = None, 597 section5: Optional[NDArray] = None, *args, **kwargs): 598 599 if np.all(section1==0): 600 try: 601 # Python >= 3.10 602 section1[5:11] = datetime.datetime.fromtimestamp(0, datetime.UTC).timetuple()[:6] 603 except(AttributeError): 604 # Python < 3.10 605 section1[5:11] = datetime.datetime.utcfromtimestamp(0).timetuple()[:6] 606 607 bases = list() 608 if section3 is None: 609 if 'gdtn' in kwargs.keys(): 610 gdtn = kwargs['gdtn'] 611 Gdt = templates.gdt_class_by_gdtn(gdtn) 612 bases.append(Gdt) 613 section3 = np.zeros((Gdt._len+5),dtype=np.int64) 614 section3[4] = gdtn 615 else: 616 raise ValueError("Must provide GRIB2 Grid Definition Template Number or section 3 array") 617 else: 618 gdtn = section3[4] 619 Gdt = templates.gdt_class_by_gdtn(gdtn) 620 bases.append(Gdt) 621 622 if section4 is None: 623 if 'pdtn' in kwargs.keys(): 624 pdtn = kwargs['pdtn'] 625 Pdt = templates.pdt_class_by_pdtn(pdtn) 626 bases.append(Pdt) 627 section4 = np.zeros((Pdt._len+2),dtype=np.int64) 628 section4[1] = pdtn 629 else: 630 raise ValueError("Must provide GRIB2 Production Definition Template Number or section 4 array") 631 else: 632 pdtn = section4[1] 633 Pdt = templates.pdt_class_by_pdtn(pdtn) 634 bases.append(Pdt) 635 636 if section5 is None: 637 if 'drtn' in kwargs.keys(): 638 drtn = kwargs['drtn'] 639 Drt = templates.drt_class_by_drtn(drtn) 640 bases.append(Drt) 641 section5 = np.zeros((Drt._len+2),dtype=np.int64) 642 section5[1] = drtn 643 else: 644 raise ValueError("Must provide GRIB2 Data Representation Template Number or section 5 array") 645 else: 646 drtn = section5[1] 647 Drt = templates.drt_class_by_drtn(drtn) 648 bases.append(Drt) 649 650 # attempt to use existing Msg class if it has already been made with gdtn,pdtn,drtn combo 651 try: 652 Msg = _msg_class_store[f"{gdtn}:{pdtn}:{drtn}"] 653 except KeyError: 654 @dataclass(init=False, repr=False) 655 class Msg(_Grib2Message, *bases): 656 pass 657 _msg_class_store[f"{gdtn}:{pdtn}:{drtn}"] = Msg 658 659 return Msg(section0, section1, section2, section3, section4, section5, *args)
Creation class for a GRIB2 message.
This class returns a dynamically-created Grib2Message object that
inherits from _Grib2Message
and grid, product, data representation
template classes according to the template numbers for the respective
sections. If section3
, section4
, or section5
are omitted, then
the appropriate keyword arguments for the template number gdtn=
,
pdtn=
, or drtn=
must be provided.
Parameters
- section0: GRIB2 section 0 array.
- section1: GRIB2 section 1 array.
- section2: Local Use section data.
- section3: GRIB2 section 3 array.
- section4: GRIB2 section 4 array.
- section5: GRIB2 section 5 array.
Returns
- Msg: A dynamically-create Grib2Message object that inherits from _Grib2Message, a grid definition template class, product definition template class, and a data representation template class.
662@dataclass 663class _Grib2Message: 664 """ 665 GRIB2 Message base class. 666 """ 667 # GRIB2 Sections 668 section0: NDArray = field(init=True,repr=False) 669 section1: NDArray = field(init=True,repr=False) 670 section2: bytes = field(init=True,repr=False) 671 section3: NDArray = field(init=True,repr=False) 672 section4: NDArray = field(init=True,repr=False) 673 section5: NDArray = field(init=True,repr=False) 674 bitMapFlag: templates.Grib2Metadata = field(init=True,repr=False,default=255) 675 676 # Section 0 looked up attributes 677 indicatorSection: NDArray = field(init=False,repr=False,default=templates.IndicatorSection()) 678 discipline: templates.Grib2Metadata = field(init=False,repr=False,default=templates.Discipline()) 679 680 # Section 1 looked up attributes 681 identificationSection: NDArray = field(init=False,repr=False,default=templates.IdentificationSection()) 682 originatingCenter: templates.Grib2Metadata = field(init=False,repr=False,default=templates.OriginatingCenter()) 683 originatingSubCenter: templates.Grib2Metadata = field(init=False,repr=False,default=templates.OriginatingSubCenter()) 684 masterTableInfo: templates.Grib2Metadata = field(init=False,repr=False,default=templates.MasterTableInfo()) 685 localTableInfo: templates.Grib2Metadata = field(init=False,repr=False,default=templates.LocalTableInfo()) 686 significanceOfReferenceTime: templates.Grib2Metadata = field(init=False,repr=False,default=templates.SignificanceOfReferenceTime()) 687 year: int = field(init=False,repr=False,default=templates.Year()) 688 month: int = field(init=False,repr=False,default=templates.Month()) 689 day: int = field(init=False,repr=False,default=templates.Day()) 690 hour: int = field(init=False,repr=False,default=templates.Hour()) 691 minute: int = field(init=False,repr=False,default=templates.Minute()) 692 second: int = field(init=False,repr=False,default=templates.Second()) 693 refDate: datetime.datetime = field(init=False,repr=False,default=templates.RefDate()) 694 productionStatus: templates.Grib2Metadata = field(init=False,repr=False,default=templates.ProductionStatus()) 695 typeOfData: templates.Grib2Metadata = field(init=False,repr=False,default=templates.TypeOfData()) 696 697 # Section 3 looked up common attributes. Other looked up attributes are available according 698 # to the Grid Definition Template. 699 gridDefinitionSection: NDArray = field(init=False,repr=False,default=templates.GridDefinitionSection()) 700 sourceOfGridDefinition: int = field(init=False,repr=False,default=templates.SourceOfGridDefinition()) 701 numberOfDataPoints: int = field(init=False,repr=False,default=templates.NumberOfDataPoints()) 702 interpretationOfListOfNumbers: templates.Grib2Metadata = field(init=False,repr=False,default=templates.InterpretationOfListOfNumbers()) 703 gridDefinitionTemplateNumber: templates.Grib2Metadata = field(init=False,repr=False,default=templates.GridDefinitionTemplateNumber()) 704 gridDefinitionTemplate: list = field(init=False,repr=False,default=templates.GridDefinitionTemplate()) 705 _earthparams: dict = field(init=False,repr=False,default=templates.EarthParams()) 706 _dxsign: float = field(init=False,repr=False,default=templates.DxSign()) 707 _dysign: float = field(init=False,repr=False,default=templates.DySign()) 708 _llscalefactor: float = field(init=False,repr=False,default=templates.LLScaleFactor()) 709 _lldivisor: float = field(init=False,repr=False,default=templates.LLDivisor()) 710 _xydivisor: float = field(init=False,repr=False,default=templates.XYDivisor()) 711 shapeOfEarth: templates.Grib2Metadata = field(init=False,repr=False,default=templates.ShapeOfEarth()) 712 earthShape: str = field(init=False,repr=False,default=templates.EarthShape()) 713 earthRadius: float = field(init=False,repr=False,default=templates.EarthRadius()) 714 earthMajorAxis: float = field(init=False,repr=False,default=templates.EarthMajorAxis()) 715 earthMinorAxis: float = field(init=False,repr=False,default=templates.EarthMinorAxis()) 716 resolutionAndComponentFlags: list = field(init=False,repr=False,default=templates.ResolutionAndComponentFlags()) 717 ny: int = field(init=False,repr=False,default=templates.Ny()) 718 nx: int = field(init=False,repr=False,default=templates.Nx()) 719 scanModeFlags: list = field(init=False,repr=False,default=templates.ScanModeFlags()) 720 projParameters: dict = field(init=False,repr=False,default=templates.ProjParameters()) 721 722 # Section 4 723 productDefinitionTemplateNumber: templates.Grib2Metadata = field(init=False,repr=False,default=templates.ProductDefinitionTemplateNumber()) 724 productDefinitionTemplate: NDArray = field(init=False,repr=False,default=templates.ProductDefinitionTemplate()) 725 726 # Section 5 looked up common attributes. Other looked up attributes are 727 # available according to the Data Representation Template. 728 numberOfPackedValues: int = field(init=False,repr=False,default=templates.NumberOfPackedValues()) 729 dataRepresentationTemplateNumber: templates.Grib2Metadata = field(init=False,repr=False,default=templates.DataRepresentationTemplateNumber()) 730 dataRepresentationTemplate: list = field(init=False,repr=False,default=templates.DataRepresentationTemplate()) 731 typeOfValues: templates.Grib2Metadata = field(init=False,repr=False,default=templates.TypeOfValues()) 732 733 def __post_init__(self): 734 """Set some attributes after init.""" 735 self._auto_nans = _AUTO_NANS 736 self._coordlist = np.zeros((0), dtype=np.float32) 737 self._data = None 738 self._deflist = np.zeros((0), dtype=np.int64) 739 self._msgnum = -1 740 self._ondiskarray = None 741 self._orig_section5 = np.copy(self.section5) 742 self._signature = self._generate_signature() 743 try: 744 self._sha1_section3 = hashlib.sha1(self.section3).hexdigest() 745 except(TypeError): 746 pass 747 self.bitMapFlag = templates.Grib2Metadata(self.bitMapFlag,table='6.0') 748 self.bitmap = None 749 750 @property 751 def _isNDFD(self): 752 """Check if GRIB2 message is from NWS NDFD""" 753 return np.all(self.section1[0:2]==[8,65535]) 754 755 @property 756 def _isAerosol(self): 757 """Check if GRIB2 message contains aerosol data""" 758 is_aero_template = self.productDefinitionTemplateNumber.value in tables.AEROSOL_PDTNS 759 is_aero_param = ((str(self.parameterCategory) == '13') | 760 (str(self.parameterCategory) == '20')) and str(self.parameterNumber) in tables.AEROSOL_PARAMS 761 # Check table 4.205 aerosol presence 762 is_aero_type = (str(self.parameterCategory) == '205' and 763 str(self.parameterNumber) == '1') 764 return is_aero_template or is_aero_param or is_aero_type 765 766 @property 767 def gdtn(self): 768 """Return Grid Definition Template Number""" 769 return self.section3[4] 770 771 772 @property 773 def gdt(self): 774 """Return Grid Definition Template.""" 775 return self.gridDefinitionTemplate 776 777 778 @property 779 def pdtn(self): 780 """Return Product Definition Template Number.""" 781 return self.section4[1] 782 783 784 @property 785 def pdt(self): 786 """Return Product Definition Template.""" 787 return self.productDefinitionTemplate 788 789 790 @property 791 def drtn(self): 792 """Return Data Representation Template Number.""" 793 return self.section5[1] 794 795 796 @property 797 def drt(self): 798 """Return Data Representation Template.""" 799 return self.dataRepresentationTemplate 800 801 802 @property 803 def pdy(self): 804 """Return the PDY ('YYYYMMDD').""" 805 return ''.join([str(i) for i in self.section1[5:8]]) 806 807 808 @property 809 def griddef(self): 810 """Return a Grib2GridDef instance for a GRIB2 message.""" 811 return Grib2GridDef.from_section3(self.section3) 812 813 814 @property 815 def lats(self): 816 """Return grid latitudes.""" 817 return self.latlons()[0] 818 819 820 @property 821 def lons(self): 822 """Return grid longitudes.""" 823 return self.latlons()[1] 824 825 @property 826 def min(self): 827 """Return minimum value of data.""" 828 return np.nanmin(self.data) 829 830 831 @property 832 def max(self): 833 """Return maximum value of data.""" 834 return np.nanmax(self.data) 835 836 837 @property 838 def mean(self): 839 """Return mean value of data.""" 840 return np.nanmean(self.data) 841 842 843 @property 844 def median(self): 845 """Return median value of data.""" 846 return np.nanmedian(self.data) 847 848 849 @property 850 def shape(self): 851 """Return shape of data.""" 852 return self.griddef.shape 853 854 855 def __repr__(self): 856 """ 857 Return an unambiguous string representation of the object. 858 859 Returns 860 ------- 861 repr 862 A string representation of the object, including information from 863 sections 0, 1, 3, 4, 5, and 6. 864 """ 865 info = '' 866 for sect in [0,1,3,4,5,6]: 867 for k,v in self.attrs_by_section(sect,values=True).items(): 868 info += f'Section {sect}: {k} = {v}\n' 869 return info 870 871 def __str__(self): 872 """ 873 Return a readable string representation of the object. 874 875 Returns 876 ------- 877 str 878 A formatted string representation of the object, including 879 selected attributes. 880 """ 881 return (f'{self._msgnum}:d={self.refDate}:{self.shortName}:' 882 f'{self.fullName} ({self.units}):{self.level}:' 883 f'{self.leadTime}') 884 885 886 def _generate_signature(self): 887 """Generature SHA-1 hash string from GRIB2 integer sections.""" 888 return hashlib.sha1(np.concatenate((self.section0,self.section1, 889 self.section3,self.section4, 890 self.section5))).hexdigest() 891 892 893 def attrs_by_section(self, sect: int, values: bool=False): 894 """ 895 Provide a tuple of attribute names for the given GRIB2 section. 896 897 Parameters 898 ---------- 899 sect 900 The GRIB2 section number. 901 values 902 Optional (default is `False`) argument to return attributes values. 903 904 Returns 905 ------- 906 attrs_by_section 907 A list of attribute names or dict of name:value pairs if `values = 908 True`. 909 """ 910 if sect in {0,1,6}: 911 attrs = templates._section_attrs[sect] 912 elif sect in {3,4,5}: 913 def _find_class_index(n): 914 _key = {3:'Grid', 4:'Product', 5:'Data'} 915 for i,c in enumerate(self.__class__.__mro__): 916 if _key[n] in c.__name__: 917 return i 918 else: 919 return [] 920 if sys.version_info.minor <= 8: 921 attrs = templates._section_attrs[sect]+\ 922 [a for a in dir(self.__class__.__mro__[_find_class_index(sect)]) if not a.startswith('_')] 923 else: 924 attrs = templates._section_attrs[sect]+\ 925 self.__class__.__mro__[_find_class_index(sect)]._attrs() 926 else: 927 attrs = [] 928 if values: 929 return {k:getattr(self,k) for k in attrs} 930 else: 931 return attrs 932 933 934 def pack(self): 935 """ 936 Pack GRIB2 section data into a binary message. 937 938 It is the user's responsibility to populate the GRIB2 section 939 information with appropriate metadata. 940 """ 941 # Create beginning of packed binary message with section 0 and 1 data. 942 self._sections = [] 943 self._msg,self._pos = g2clib.grib2_create(self.indicatorSection[2:4],self.identificationSection) 944 self._sections += [0,1] 945 946 # Add section 2 if present. 947 if isinstance(self.section2,bytes) and len(self.section2) > 0: 948 self._msg,self._pos = g2clib.grib2_addlocal(self._msg,self.section2) 949 self._sections.append(2) 950 951 # Add section 3. 952 self.section3[1] = self.nx * self.ny 953 self._msg,self._pos = g2clib.grib2_addgrid(self._msg,self.gridDefinitionSection, 954 self.gridDefinitionTemplate, 955 self._deflist) 956 self._sections.append(3) 957 958 # Prepare data. 959 if self._data is None: 960 if self._ondiskarray is None: 961 raise ValueError("Grib2Message object has no data, thus it cannot be packed.") 962 field = np.copy(self.data) 963 if self.scanModeFlags is not None: 964 if self.scanModeFlags[3]: 965 fieldsave = field.astype('f') # Casting makes a copy 966 field[1::2,:] = fieldsave[1::2,::-1] 967 fld = field.astype('f') 968 969 # Prepare bitmap, if necessary 970 bitmapflag = self.bitMapFlag.value 971 if bitmapflag == 0: 972 if self.bitmap is not None: 973 bmap = np.ravel(self.bitmap).astype(DEFAULT_NUMPY_INT) 974 else: 975 bmap = np.ravel(np.where(np.isnan(fld),0,1)).astype(DEFAULT_NUMPY_INT) 976 else: 977 bmap = None 978 979 # Prepare data for packing if nans are present 980 fld = np.ravel(fld) 981 if bitmapflag in {0,254}: 982 fld = np.where(np.isnan(fld),0,fld) 983 else: 984 if np.isnan(fld).any(): 985 if hasattr(self,'priMissingValue'): 986 fld = np.where(np.isnan(fld),self.priMissingValue,fld) 987 if hasattr(self,'_missvalmap'): 988 if hasattr(self,'priMissingValue'): 989 fld = np.where(self._missvalmap==1,self.priMissingValue,fld) 990 if hasattr(self,'secMissingValue'): 991 fld = np.where(self._missvalmap==2,self.secMissingValue,fld) 992 993 # Add sections 4, 5, 6, and 7. 994 self._msg,self._pos = g2clib.grib2_addfield(self._msg,self.pdtn, 995 self.productDefinitionTemplate, 996 self._coordlist, 997 self.drtn, 998 self.dataRepresentationTemplate, 999 fld, 1000 bitmapflag, 1001 bmap) 1002 self._sections.append(4) 1003 self._sections.append(5) 1004 self._sections.append(6) 1005 self._sections.append(7) 1006 1007 # Finalize GRIB2 message with section 8. 1008 self._msg, self._pos = g2clib.grib2_end(self._msg) 1009 self._sections.append(8) 1010 self.section0[-1] = len(self._msg) 1011 1012 1013 @property 1014 def data(self) -> np.array: 1015 """Access the unpacked data values.""" 1016 if self._data is None: 1017 if self._auto_nans != _AUTO_NANS: 1018 self._data = self._ondiskarray 1019 self._data = np.asarray(self._ondiskarray) 1020 return self._data 1021 1022 1023 @data.setter 1024 def data(self, data): 1025 if not isinstance(data, np.ndarray): 1026 raise ValueError('Grib2Message data only supports numpy arrays') 1027 self._data = data 1028 1029 1030 def flush_data(self): 1031 """ 1032 Flush the unpacked data values from the Grib2Message object. 1033 1034 Note: If the Grib2Message object was constructed from "scratch" (i.e. 1035 not read from file), this method will remove the data array from 1036 the object and it cannot be recovered. 1037 """ 1038 self._data = None 1039 self.bitmap = None 1040 1041 1042 def __getitem__(self, item): 1043 return self.data[item] 1044 1045 1046 def __setitem__(self, item): 1047 raise NotImplementedError('assignment of data not supported via setitem') 1048 1049 1050 def latlons(self, *args, **kwrgs): 1051 """Alias for `grib2io.Grib2Message.grid` method.""" 1052 return self.grid(*args, **kwrgs) 1053 1054 1055 def grid(self, unrotate: bool=True): 1056 """ 1057 Return lats,lons (in degrees) of grid. 1058 1059 Currently can handle reg. lat/lon,cglobal Gaussian, mercator, 1060 stereographic, lambert conformal, albers equal-area, space-view and 1061 azimuthal equidistant grids. 1062 1063 Parameters 1064 ---------- 1065 unrotate 1066 If `True` [DEFAULT], and grid is rotated lat/lon, then unrotate the 1067 grid, otherwise `False`, do not. 1068 1069 Returns 1070 ------- 1071 lats, lons : numpy.ndarray 1072 Returns two numpy.ndarrays with dtype=numpy.float32 of grid 1073 latitudes and longitudes in units of degrees. 1074 """ 1075 if self._sha1_section3 in _latlon_datastore.keys(): 1076 return (_latlon_datastore[self._sha1_section3]['latitude'], 1077 _latlon_datastore[self._sha1_section3]['longitude']) 1078 gdtn = self.gridDefinitionTemplateNumber.value 1079 gdtmpl = self.gridDefinitionTemplate 1080 reggrid = self.gridDefinitionSection[2] == 0 # This means regular 2-d grid 1081 if gdtn == 0: 1082 # Regular lat/lon grid 1083 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1084 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1085 dlon = self.gridlengthXDirection 1086 dlat = self.gridlengthYDirection 1087 if lon2 < lon1 and dlon < 0: lon1 = -lon1 1088 lats = np.linspace(lat1,lat2,self.ny) 1089 if reggrid: 1090 lons = np.linspace(lon1,lon2,self.nx) 1091 else: 1092 lons = np.linspace(lon1,lon2,self.ny*2) 1093 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1094 elif gdtn == 1: # Rotated Lat/Lon grid 1095 pj = pyproj.Proj(self.projParameters) 1096 lat1,lon1 = self.latitudeFirstGridpoint,self.longitudeFirstGridpoint 1097 lat2,lon2 = self.latitudeLastGridpoint,self.longitudeLastGridpoint 1098 if lon1 > 180.0: lon1 -= 360.0 1099 if lon2 > 180.0: lon2 -= 360.0 1100 lats = np.linspace(lat1,lat2,self.ny) 1101 lons = np.linspace(lon1,lon2,self.nx) 1102 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1103 if unrotate: 1104 from grib2io.utils import rotated_grid 1105 lats,lons = rotated_grid.unrotate(lats,lons,self.anglePoleRotation, 1106 self.latitudeSouthernPole, 1107 self.longitudeSouthernPole) 1108 elif gdtn == 40: # Gaussian grid (only works for global!) 1109 from grib2io.utils.gauss_grid import gaussian_latitudes 1110 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1111 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1112 nlats = self.ny 1113 if not reggrid: # Reduced Gaussian grid. 1114 nlons = 2*nlats 1115 dlon = 360./nlons 1116 else: 1117 nlons = self.nx 1118 dlon = self.gridlengthXDirection 1119 lons = np.linspace(lon1,lon2,nlons) 1120 # Compute Gaussian lats (north to south) 1121 lats = gaussian_latitudes(nlats) 1122 if lat1 < lat2: # reverse them if necessary 1123 lats = lats[::-1] 1124 lons,lats = np.meshgrid(lons,lats) 1125 elif gdtn in {10,20,30,31,110}: 1126 # Mercator, Lambert Conformal, Stereographic, Albers Equal Area, 1127 # Azimuthal Equidistant 1128 dx,dy = self.gridlengthXDirection, self.gridlengthYDirection 1129 lon1,lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1130 pj = pyproj.Proj(self.projParameters) 1131 llcrnrx, llcrnry = pj(lon1,lat1) 1132 x = llcrnrx+dx*np.arange(self.nx) 1133 y = llcrnry+dy*np.arange(self.ny) 1134 x,y = np.meshgrid(x, y) 1135 lons,lats = pj(x, y, inverse=True) 1136 elif gdtn == 90: 1137 # Satellite Projection 1138 dx = self.gridlengthXDirection 1139 dy = self.gridlengthYDirection 1140 pj = pyproj.Proj(self.projParameters) 1141 x = dx*np.indices((self.ny,self.nx),'f')[1,:,:] 1142 x -= 0.5*x.max() 1143 y = dy*np.indices((self.ny,self.nx),'f')[0,:,:] 1144 y -= 0.5*y.max() 1145 lons,lats = pj(x,y,inverse=True) 1146 # Set lons,lats to 1.e30 where undefined 1147 abslons = np.fabs(lons) 1148 abslats = np.fabs(lats) 1149 lons = np.where(abslons < 1.e20, lons, 1.e30) 1150 lats = np.where(abslats < 1.e20, lats, 1.e30) 1151 elif gdtn == 32769: 1152 # Special NCEP Grid, Rotated Lat/Lon, Arakawa E-Grid (Non-Staggered) 1153 from grib2io.utils import arakawa_rotated_grid 1154 from grib2io.utils.rotated_grid import DEG2RAD 1155 di, dj = 0.0, 0.0 1156 do_180 = False 1157 idir = 1 if self.scanModeFlags[0] == 0 else -1 1158 jdir = -1 if self.scanModeFlags[1] == 0 else 1 1159 grid_oriented = 0 if self.resolutionAndComponentFlags[4] == 0 else 1 1160 do_rot = 1 if grid_oriented == 1 else 0 1161 la1 = self.latitudeFirstGridpoint 1162 lo1 = self.longitudeFirstGridpoint 1163 clon = self.longitudeCenterGridpoint 1164 clat = self.latitudeCenterGridpoint 1165 lasp = clat - 90.0 1166 losp = clon 1167 llat, llon = arakawa_rotated_grid.ll2rot(la1,lo1,lasp,losp) 1168 la2, lo2 = arakawa_rotated_grid.rot2ll(-llat,-llon,lasp,losp) 1169 rlat = -llat 1170 rlon = -llon 1171 if self.nx == 1: 1172 di = 0.0 1173 elif idir == 1: 1174 ti = rlon 1175 while ti < llon: 1176 ti += 360.0 1177 di = (ti - llon)/float(self.nx-1) 1178 else: 1179 ti = llon 1180 while ti < rlon: 1181 ti += 360.0 1182 di = (ti - rlon)/float(self.nx-1) 1183 if self.ny == 1: 1184 dj = 0.0 1185 else: 1186 dj = (rlat - llat)/float(self.ny-1) 1187 if dj < 0.0: 1188 dj = -dj 1189 if idir == 1: 1190 if llon > rlon: 1191 llon -= 360.0 1192 if llon < 0 and rlon > 0: 1193 do_180 = True 1194 else: 1195 if rlon > llon: 1196 rlon -= 360.0 1197 if rlon < 0 and llon > 0: 1198 do_180 = True 1199 xlat1d = llat + (np.arange(self.ny)*jdir*dj) 1200 xlon1d = llon + (np.arange(self.nx)*idir*di) 1201 xlons, xlats = np.meshgrid(xlon1d,xlat1d) 1202 rot2ll_vectorized = np.vectorize(arakawa_rotated_grid.rot2ll) 1203 lats, lons = rot2ll_vectorized(xlats,xlons,lasp,losp) 1204 if do_180: 1205 lons = np.where(lons>180.0,lons-360.0,lons) 1206 vector_rotation_angles_vectorized = np.vectorize(arakawa_rotated_grid.vector_rotation_angles) 1207 rots = vector_rotation_angles_vectorized(lats, lons, clat, losp, xlats) 1208 del xlat1d, xlon1d, xlats, xlons 1209 else: 1210 raise ValueError('Unsupported grid') 1211 1212 _latlon_datastore[self._sha1_section3] = dict(latitude=lats,longitude=lons) 1213 try: 1214 _latlon_datastore[self._sha1_section3]['vector_rotation_angles'] = rots 1215 except(NameError): 1216 pass 1217 1218 return lats, lons 1219 1220 1221 def map_keys(self): 1222 """ 1223 Unpack data grid replacing integer values with strings. 1224 1225 These types of fields are categorical or classifications where data 1226 values do not represent an observable or predictable physical quantity. 1227 An example of such a field would be [Dominant Precipitation Type - 1228 DPTYPE](https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table4-201.shtml) 1229 1230 Returns 1231 ------- 1232 map_keys 1233 numpy.ndarray of string values per element. 1234 """ 1235 hold_auto_nans = _AUTO_NANS 1236 set_auto_nans(False) 1237 1238 if (np.all(self.section1[0:2] == [7, 14]) and self.shortName == "PWTHER") or ( 1239 self._isNDFD and self.shortName in {"WX", "WWA"} 1240 ): 1241 keys = utils.decode_wx_strings(self.section2) 1242 if hasattr(self,'priMissingValue') and self.priMissingValue not in [None,0]: 1243 keys[int(self.priMissingValue)] = 'Missing' 1244 if hasattr(self,'secMissingValue') and self.secMissingValue not in [None,0]: 1245 keys[int(self.secMissingValue)] = 'Missing' 1246 u,inv = np.unique(self.data,return_inverse=True) 1247 fld = np.array([keys[x] for x in u])[inv].reshape(self.data.shape) 1248 else: 1249 # For data whose units are defined in a code table (i.e. classification or mask) 1250 tblname = re.findall(r'\d\.\d+',self.units,re.IGNORECASE)[0] 1251 fld = self.data.astype(np.int32).astype(str) 1252 tbl = tables.get_table(tblname,expand=True) 1253 for val in np.unique(fld): 1254 fld = np.where(fld==val,tbl[val],fld) 1255 set_auto_nans(hold_auto_nans) 1256 return fld 1257 1258 1259 def to_bytes(self, validate: bool=True): 1260 """ 1261 Return packed GRIB2 message in bytes format. 1262 1263 This will be useful for exporting data in non-file formats. For example, 1264 can be used to output grib data directly to S3 using the boto3 client 1265 without the need to write a temporary file to upload first. 1266 1267 Parameters 1268 ---------- 1269 validate: default=True 1270 If `True` (DEFAULT), validates first/last four bytes for proper 1271 formatting, else returns None. If `False`, message is output as is. 1272 1273 Returns 1274 ------- 1275 to_bytes 1276 Returns GRIB2 formatted message as bytes. 1277 """ 1278 if hasattr(self,'_msg'): 1279 if validate: 1280 if self.validate(): 1281 return self._msg 1282 else: 1283 return None 1284 else: 1285 return self._msg 1286 else: 1287 return None 1288 1289 1290 def interpolate(self, method, grid_def_out, method_options=None, drtn=None, 1291 num_threads=1): 1292 """ 1293 Grib2Message Interpolator 1294 1295 Performs spatial interpolation via the [NCEPLIBS-ip 1296 library](https://github.com/NOAA-EMC/NCEPLIBS-ip). This interpolate 1297 method only supports scalar interpolation. If you need to perform 1298 vector interpolation, use the module-level `grib2io.interpolate` 1299 function. 1300 1301 Parameters 1302 ---------- 1303 method 1304 Interpolate method to use. This can either be an integer or string 1305 using the following mapping: 1306 1307 | Interpolate Scheme | Integer Value | 1308 | :---: | :---: | 1309 | 'bilinear' | 0 | 1310 | 'bicubic' | 1 | 1311 | 'neighbor' | 2 | 1312 | 'budget' | 3 | 1313 | 'spectral' | 4 | 1314 | 'neighbor-budget' | 6 | 1315 1316 grid_def_out : grib2io.Grib2GridDef 1317 Grib2GridDef object of the output grid. 1318 method_options : list of ints, optional 1319 Interpolation options. See the NCEPLIBS-ip documentation for 1320 more information on how these are used. 1321 drtn 1322 Data Representation Template to be used for the returned 1323 interpolated GRIB2 message. When `None`, the data representation 1324 template of the source GRIB2 message is used. Once again, it is the 1325 user's responsibility to properly set the Data Representation 1326 Template attributes. 1327 num_threads : int, optional 1328 Number of OpenMP threads to use for interpolation. The default 1329 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1330 was not built with OpenMP, then this keyword argument and value 1331 will have no impact. 1332 1333 Returns 1334 ------- 1335 interpolate 1336 If interpolating to a grid, a new Grib2Message object is returned. 1337 The GRIB2 metadata of the new Grib2Message object is identical to 1338 the input except where required to be different because of the new 1339 grid specs and possibly a new data representation template. 1340 1341 If interpolating to station points, the interpolated data values are 1342 returned as a numpy.ndarray. 1343 """ 1344 section0 = self.section0 1345 section0[-1] = 0 1346 gds = [0, grid_def_out.npoints, 0, 255, grid_def_out.gdtn] 1347 section3 = np.concatenate((gds,grid_def_out.gdt)) 1348 drtn = self.drtn if drtn is None else drtn 1349 1350 msg = Grib2Message(section0,self.section1,self.section2,section3, 1351 self.section4,None,self.bitMapFlag.value,drtn=drtn) 1352 1353 msg._msgnum = -1 1354 msg._deflist = self._deflist 1355 msg._coordlist = self._coordlist 1356 shape = (msg.ny,msg.nx) 1357 ndim = 2 1358 if msg.typeOfValues == 0: 1359 dtype = 'float32' 1360 elif msg.typeOfValues == 1: 1361 dtype = 'int32' 1362 msg._data = interpolate(self.data,method,Grib2GridDef.from_section3(self.section3),grid_def_out, 1363 method_options=method_options,num_threads=num_threads).reshape(msg.ny,msg.nx) 1364 msg.section5[0] = grid_def_out.npoints 1365 return msg 1366 1367 1368 def subset(self, lats, lons): 1369 """ 1370 Return a spatial subset. 1371 1372 Currently only supports regular grids of the following types: 1373 1374 | Grid Type | gdtn | 1375 | :---: | :---: | 1376 | Latitude/Longitude, Equidistant Cylindrical, or Plate Carree | 0 | 1377 | Rotated Latitude/Longitude | 1 | 1378 | Mercator | 10 | 1379 | Polar Stereographic | 20 | 1380 | Lambert Conformal | 30 | 1381 | Albers Equal-Area | 31 | 1382 | Gaussian Latitude/Longitude | 40 | 1383 | Equatorial Azimuthal Equidistant Projection | 110 | 1384 1385 Parameters 1386 ---------- 1387 lats 1388 List or tuple of latitudes. The minimum and maximum latitudes will 1389 be used to define the southern and northern boundaries. 1390 1391 The order of the latitudes is not important. The function will 1392 determine which is the minimum and maximum. 1393 1394 The latitudes should be in decimal degrees with 0.0 at the equator, 1395 positive values in the northern hemisphere increasing to 90, and 1396 negative values in the southern hemisphere decreasing to -90. 1397 lons 1398 List or tuple of longitudes. The minimum and maximum longitudes 1399 will be used to define the western and eastern boundaries. 1400 1401 The order of the longitudes is not important. The function will 1402 determine which is the minimum and maximum. 1403 1404 GRIB2 longitudes should be in decimal degrees with 0.0 at the prime 1405 meridian, positive values increasing eastward to 360. There are no 1406 negative GRIB2 longitudes. 1407 1408 The typical west longitudes that start at 0.0 at the prime meridian 1409 and decrease to -180 westward, are converted to GRIB2 longitudes by 1410 '360 - (absolute value of the west longitude)' where typical 1411 eastern longitudes are unchanged as GRIB2 longitudes. 1412 1413 Returns 1414 ------- 1415 subset 1416 A spatial subset of a GRIB2 message. 1417 """ 1418 if self.gdtn not in [0, 1, 10, 20, 30, 31, 40, 110]: 1419 raise ValueError( 1420 """ 1421 1422Subset only works for 1423 Latitude/Longitude, Equidistant Cylindrical, or Plate Carree (gdtn=0) 1424 Rotated Latitude/Longitude (gdtn=1) 1425 Mercator (gdtn=10) 1426 Polar Stereographic (gdtn=20) 1427 Lambert Conformal (gdtn=30) 1428 Albers Equal-Area (gdtn=31) 1429 Gaussian Latitude/Longitude (gdtn=40) 1430 Equatorial Azimuthal Equidistant Projection (gdtn=110) 1431 1432""" 1433 ) 1434 1435 if self.nx == 0 or self.ny == 0: 1436 raise ValueError( 1437 """ 1438 1439Subset only works for regular grids. 1440 1441""" 1442 ) 1443 1444 newmsg = Grib2Message( 1445 np.copy(self.section0), 1446 np.copy(self.section1), 1447 np.copy(self.section2), 1448 np.copy(self.section3), 1449 np.copy(self.section4), 1450 np.copy(self.section5), 1451 ) 1452 1453 msglats, msglons = self.grid() 1454 1455 la1 = np.max(lats) 1456 lo1 = np.min(lons) 1457 la2 = np.min(lats) 1458 lo2 = np.max(lons) 1459 1460 # Find the indices of the first and last grid points to the nearest 1461 # lat/lon values in the grid. 1462 first_lat = np.abs(msglats - la1) 1463 first_lon = np.abs(msglons - lo1) 1464 max_idx = np.maximum(first_lat, first_lon) 1465 first_j, first_i = np.where(max_idx == np.min(max_idx)) 1466 1467 last_lat = np.abs(msglats - la2) 1468 last_lon = np.abs(msglons - lo2) 1469 max_idx = np.maximum(last_lat, last_lon) 1470 last_j, last_i = np.where(max_idx == np.min(max_idx)) 1471 1472 setattr(newmsg, "latitudeFirstGridpoint", msglats[first_j[0], first_i[0]]) 1473 setattr(newmsg, "longitudeFirstGridpoint", msglons[first_j[0], first_i[0]]) 1474 setattr(newmsg, "nx", np.abs(first_i[0] - last_i[0])) 1475 setattr(newmsg, "ny", np.abs(first_j[0] - last_j[0])) 1476 1477 # Set *LastGridpoint attributes even if only used for gdtn=[0, 1, 40]. 1478 # This information is used to subset xarray datasets and even though 1479 # unnecessary for some supported grid types, it won't affect a grib2io 1480 # message to set them. 1481 setattr(newmsg, "latitudeLastGridpoint", msglats[last_j[0], last_i[0]]) 1482 setattr(newmsg, "longitudeLastGridpoint", msglons[last_j[0], last_i[0]]) 1483 1484 setattr( 1485 newmsg, 1486 "data", 1487 self.data[ 1488 min(first_j[0], last_j[0]) : max(first_j[0], last_j[0]), 1489 min(first_i[0], last_i[0]) : max(first_i[0], last_i[0]), 1490 ].copy(), 1491 ) 1492 1493 # Need to set the newmsg._sha1_section3 to a blank string so the grid 1494 # method ignores the cached lat/lon values. This will force the grid 1495 # method to recompute the lat/lon values for the subsetted grid. 1496 newmsg._sha1_section3 = "" 1497 newmsg.grid() 1498 1499 return newmsg 1500 1501 def validate(self): 1502 """ 1503 Validate a complete GRIB2 message. 1504 1505 The g2c library does its own internal validation when g2_gribend() is called, but 1506 we will check in grib2io also. The validation checks if the first 4 bytes in 1507 self._msg is 'GRIB' and '7777' as the last 4 bytes and that the message length in 1508 section 0 equals the length of the packed message. 1509 1510 Returns 1511 ------- 1512 `True` if the packed GRIB2 message is complete and well-formed, `False` otherwise. 1513 """ 1514 valid = False 1515 if hasattr(self,'_msg'): 1516 if self._msg[0:4]+self._msg[-4:] == b'GRIB7777': 1517 if self.section0[-1] == len(self._msg): 1518 valid = True 1519 return valid
GRIB2 Message base class.
GRIB2 Section 1, Identification Section
GRIB2 Section 3, Grid Definition Section
Product Definition Template
766 @property 767 def gdtn(self): 768 """Return Grid Definition Template Number""" 769 return self.section3[4]
Return Grid Definition Template Number
772 @property 773 def gdt(self): 774 """Return Grid Definition Template.""" 775 return self.gridDefinitionTemplate
Return Grid Definition Template.
778 @property 779 def pdtn(self): 780 """Return Product Definition Template Number.""" 781 return self.section4[1]
Return Product Definition Template Number.
784 @property 785 def pdt(self): 786 """Return Product Definition Template.""" 787 return self.productDefinitionTemplate
Return Product Definition Template.
790 @property 791 def drtn(self): 792 """Return Data Representation Template Number.""" 793 return self.section5[1]
Return Data Representation Template Number.
796 @property 797 def drt(self): 798 """Return Data Representation Template.""" 799 return self.dataRepresentationTemplate
Return Data Representation Template.
802 @property 803 def pdy(self): 804 """Return the PDY ('YYYYMMDD').""" 805 return ''.join([str(i) for i in self.section1[5:8]])
Return the PDY ('YYYYMMDD').
808 @property 809 def griddef(self): 810 """Return a Grib2GridDef instance for a GRIB2 message.""" 811 return Grib2GridDef.from_section3(self.section3)
Return a Grib2GridDef instance for a GRIB2 message.
825 @property 826 def min(self): 827 """Return minimum value of data.""" 828 return np.nanmin(self.data)
Return minimum value of data.
831 @property 832 def max(self): 833 """Return maximum value of data.""" 834 return np.nanmax(self.data)
Return maximum value of data.
837 @property 838 def mean(self): 839 """Return mean value of data.""" 840 return np.nanmean(self.data)
Return mean value of data.
843 @property 844 def median(self): 845 """Return median value of data.""" 846 return np.nanmedian(self.data)
Return median value of data.
893 def attrs_by_section(self, sect: int, values: bool=False): 894 """ 895 Provide a tuple of attribute names for the given GRIB2 section. 896 897 Parameters 898 ---------- 899 sect 900 The GRIB2 section number. 901 values 902 Optional (default is `False`) argument to return attributes values. 903 904 Returns 905 ------- 906 attrs_by_section 907 A list of attribute names or dict of name:value pairs if `values = 908 True`. 909 """ 910 if sect in {0,1,6}: 911 attrs = templates._section_attrs[sect] 912 elif sect in {3,4,5}: 913 def _find_class_index(n): 914 _key = {3:'Grid', 4:'Product', 5:'Data'} 915 for i,c in enumerate(self.__class__.__mro__): 916 if _key[n] in c.__name__: 917 return i 918 else: 919 return [] 920 if sys.version_info.minor <= 8: 921 attrs = templates._section_attrs[sect]+\ 922 [a for a in dir(self.__class__.__mro__[_find_class_index(sect)]) if not a.startswith('_')] 923 else: 924 attrs = templates._section_attrs[sect]+\ 925 self.__class__.__mro__[_find_class_index(sect)]._attrs() 926 else: 927 attrs = [] 928 if values: 929 return {k:getattr(self,k) for k in attrs} 930 else: 931 return attrs
Provide a tuple of attribute names for the given GRIB2 section.
Parameters
- sect: The GRIB2 section number.
- values: Optional (default is
False
) argument to return attributes values.
Returns
- attrs_by_section: A list of attribute names or dict of name:value pairs if
values = True
.
934 def pack(self): 935 """ 936 Pack GRIB2 section data into a binary message. 937 938 It is the user's responsibility to populate the GRIB2 section 939 information with appropriate metadata. 940 """ 941 # Create beginning of packed binary message with section 0 and 1 data. 942 self._sections = [] 943 self._msg,self._pos = g2clib.grib2_create(self.indicatorSection[2:4],self.identificationSection) 944 self._sections += [0,1] 945 946 # Add section 2 if present. 947 if isinstance(self.section2,bytes) and len(self.section2) > 0: 948 self._msg,self._pos = g2clib.grib2_addlocal(self._msg,self.section2) 949 self._sections.append(2) 950 951 # Add section 3. 952 self.section3[1] = self.nx * self.ny 953 self._msg,self._pos = g2clib.grib2_addgrid(self._msg,self.gridDefinitionSection, 954 self.gridDefinitionTemplate, 955 self._deflist) 956 self._sections.append(3) 957 958 # Prepare data. 959 if self._data is None: 960 if self._ondiskarray is None: 961 raise ValueError("Grib2Message object has no data, thus it cannot be packed.") 962 field = np.copy(self.data) 963 if self.scanModeFlags is not None: 964 if self.scanModeFlags[3]: 965 fieldsave = field.astype('f') # Casting makes a copy 966 field[1::2,:] = fieldsave[1::2,::-1] 967 fld = field.astype('f') 968 969 # Prepare bitmap, if necessary 970 bitmapflag = self.bitMapFlag.value 971 if bitmapflag == 0: 972 if self.bitmap is not None: 973 bmap = np.ravel(self.bitmap).astype(DEFAULT_NUMPY_INT) 974 else: 975 bmap = np.ravel(np.where(np.isnan(fld),0,1)).astype(DEFAULT_NUMPY_INT) 976 else: 977 bmap = None 978 979 # Prepare data for packing if nans are present 980 fld = np.ravel(fld) 981 if bitmapflag in {0,254}: 982 fld = np.where(np.isnan(fld),0,fld) 983 else: 984 if np.isnan(fld).any(): 985 if hasattr(self,'priMissingValue'): 986 fld = np.where(np.isnan(fld),self.priMissingValue,fld) 987 if hasattr(self,'_missvalmap'): 988 if hasattr(self,'priMissingValue'): 989 fld = np.where(self._missvalmap==1,self.priMissingValue,fld) 990 if hasattr(self,'secMissingValue'): 991 fld = np.where(self._missvalmap==2,self.secMissingValue,fld) 992 993 # Add sections 4, 5, 6, and 7. 994 self._msg,self._pos = g2clib.grib2_addfield(self._msg,self.pdtn, 995 self.productDefinitionTemplate, 996 self._coordlist, 997 self.drtn, 998 self.dataRepresentationTemplate, 999 fld, 1000 bitmapflag, 1001 bmap) 1002 self._sections.append(4) 1003 self._sections.append(5) 1004 self._sections.append(6) 1005 self._sections.append(7) 1006 1007 # Finalize GRIB2 message with section 8. 1008 self._msg, self._pos = g2clib.grib2_end(self._msg) 1009 self._sections.append(8) 1010 self.section0[-1] = len(self._msg)
Pack GRIB2 section data into a binary message.
It is the user's responsibility to populate the GRIB2 section information with appropriate metadata.
1013 @property 1014 def data(self) -> np.array: 1015 """Access the unpacked data values.""" 1016 if self._data is None: 1017 if self._auto_nans != _AUTO_NANS: 1018 self._data = self._ondiskarray 1019 self._data = np.asarray(self._ondiskarray) 1020 return self._data
Access the unpacked data values.
1030 def flush_data(self): 1031 """ 1032 Flush the unpacked data values from the Grib2Message object. 1033 1034 Note: If the Grib2Message object was constructed from "scratch" (i.e. 1035 not read from file), this method will remove the data array from 1036 the object and it cannot be recovered. 1037 """ 1038 self._data = None 1039 self.bitmap = None
Flush the unpacked data values from the Grib2Message object.
Note: If the Grib2Message object was constructed from "scratch" (i.e. not read from file), this method will remove the data array from the object and it cannot be recovered.
1050 def latlons(self, *args, **kwrgs): 1051 """Alias for `grib2io.Grib2Message.grid` method.""" 1052 return self.grid(*args, **kwrgs)
Alias for grib2io.Grib2Message.grid
method.
1055 def grid(self, unrotate: bool=True): 1056 """ 1057 Return lats,lons (in degrees) of grid. 1058 1059 Currently can handle reg. lat/lon,cglobal Gaussian, mercator, 1060 stereographic, lambert conformal, albers equal-area, space-view and 1061 azimuthal equidistant grids. 1062 1063 Parameters 1064 ---------- 1065 unrotate 1066 If `True` [DEFAULT], and grid is rotated lat/lon, then unrotate the 1067 grid, otherwise `False`, do not. 1068 1069 Returns 1070 ------- 1071 lats, lons : numpy.ndarray 1072 Returns two numpy.ndarrays with dtype=numpy.float32 of grid 1073 latitudes and longitudes in units of degrees. 1074 """ 1075 if self._sha1_section3 in _latlon_datastore.keys(): 1076 return (_latlon_datastore[self._sha1_section3]['latitude'], 1077 _latlon_datastore[self._sha1_section3]['longitude']) 1078 gdtn = self.gridDefinitionTemplateNumber.value 1079 gdtmpl = self.gridDefinitionTemplate 1080 reggrid = self.gridDefinitionSection[2] == 0 # This means regular 2-d grid 1081 if gdtn == 0: 1082 # Regular lat/lon grid 1083 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1084 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1085 dlon = self.gridlengthXDirection 1086 dlat = self.gridlengthYDirection 1087 if lon2 < lon1 and dlon < 0: lon1 = -lon1 1088 lats = np.linspace(lat1,lat2,self.ny) 1089 if reggrid: 1090 lons = np.linspace(lon1,lon2,self.nx) 1091 else: 1092 lons = np.linspace(lon1,lon2,self.ny*2) 1093 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1094 elif gdtn == 1: # Rotated Lat/Lon grid 1095 pj = pyproj.Proj(self.projParameters) 1096 lat1,lon1 = self.latitudeFirstGridpoint,self.longitudeFirstGridpoint 1097 lat2,lon2 = self.latitudeLastGridpoint,self.longitudeLastGridpoint 1098 if lon1 > 180.0: lon1 -= 360.0 1099 if lon2 > 180.0: lon2 -= 360.0 1100 lats = np.linspace(lat1,lat2,self.ny) 1101 lons = np.linspace(lon1,lon2,self.nx) 1102 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1103 if unrotate: 1104 from grib2io.utils import rotated_grid 1105 lats,lons = rotated_grid.unrotate(lats,lons,self.anglePoleRotation, 1106 self.latitudeSouthernPole, 1107 self.longitudeSouthernPole) 1108 elif gdtn == 40: # Gaussian grid (only works for global!) 1109 from grib2io.utils.gauss_grid import gaussian_latitudes 1110 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1111 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1112 nlats = self.ny 1113 if not reggrid: # Reduced Gaussian grid. 1114 nlons = 2*nlats 1115 dlon = 360./nlons 1116 else: 1117 nlons = self.nx 1118 dlon = self.gridlengthXDirection 1119 lons = np.linspace(lon1,lon2,nlons) 1120 # Compute Gaussian lats (north to south) 1121 lats = gaussian_latitudes(nlats) 1122 if lat1 < lat2: # reverse them if necessary 1123 lats = lats[::-1] 1124 lons,lats = np.meshgrid(lons,lats) 1125 elif gdtn in {10,20,30,31,110}: 1126 # Mercator, Lambert Conformal, Stereographic, Albers Equal Area, 1127 # Azimuthal Equidistant 1128 dx,dy = self.gridlengthXDirection, self.gridlengthYDirection 1129 lon1,lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1130 pj = pyproj.Proj(self.projParameters) 1131 llcrnrx, llcrnry = pj(lon1,lat1) 1132 x = llcrnrx+dx*np.arange(self.nx) 1133 y = llcrnry+dy*np.arange(self.ny) 1134 x,y = np.meshgrid(x, y) 1135 lons,lats = pj(x, y, inverse=True) 1136 elif gdtn == 90: 1137 # Satellite Projection 1138 dx = self.gridlengthXDirection 1139 dy = self.gridlengthYDirection 1140 pj = pyproj.Proj(self.projParameters) 1141 x = dx*np.indices((self.ny,self.nx),'f')[1,:,:] 1142 x -= 0.5*x.max() 1143 y = dy*np.indices((self.ny,self.nx),'f')[0,:,:] 1144 y -= 0.5*y.max() 1145 lons,lats = pj(x,y,inverse=True) 1146 # Set lons,lats to 1.e30 where undefined 1147 abslons = np.fabs(lons) 1148 abslats = np.fabs(lats) 1149 lons = np.where(abslons < 1.e20, lons, 1.e30) 1150 lats = np.where(abslats < 1.e20, lats, 1.e30) 1151 elif gdtn == 32769: 1152 # Special NCEP Grid, Rotated Lat/Lon, Arakawa E-Grid (Non-Staggered) 1153 from grib2io.utils import arakawa_rotated_grid 1154 from grib2io.utils.rotated_grid import DEG2RAD 1155 di, dj = 0.0, 0.0 1156 do_180 = False 1157 idir = 1 if self.scanModeFlags[0] == 0 else -1 1158 jdir = -1 if self.scanModeFlags[1] == 0 else 1 1159 grid_oriented = 0 if self.resolutionAndComponentFlags[4] == 0 else 1 1160 do_rot = 1 if grid_oriented == 1 else 0 1161 la1 = self.latitudeFirstGridpoint 1162 lo1 = self.longitudeFirstGridpoint 1163 clon = self.longitudeCenterGridpoint 1164 clat = self.latitudeCenterGridpoint 1165 lasp = clat - 90.0 1166 losp = clon 1167 llat, llon = arakawa_rotated_grid.ll2rot(la1,lo1,lasp,losp) 1168 la2, lo2 = arakawa_rotated_grid.rot2ll(-llat,-llon,lasp,losp) 1169 rlat = -llat 1170 rlon = -llon 1171 if self.nx == 1: 1172 di = 0.0 1173 elif idir == 1: 1174 ti = rlon 1175 while ti < llon: 1176 ti += 360.0 1177 di = (ti - llon)/float(self.nx-1) 1178 else: 1179 ti = llon 1180 while ti < rlon: 1181 ti += 360.0 1182 di = (ti - rlon)/float(self.nx-1) 1183 if self.ny == 1: 1184 dj = 0.0 1185 else: 1186 dj = (rlat - llat)/float(self.ny-1) 1187 if dj < 0.0: 1188 dj = -dj 1189 if idir == 1: 1190 if llon > rlon: 1191 llon -= 360.0 1192 if llon < 0 and rlon > 0: 1193 do_180 = True 1194 else: 1195 if rlon > llon: 1196 rlon -= 360.0 1197 if rlon < 0 and llon > 0: 1198 do_180 = True 1199 xlat1d = llat + (np.arange(self.ny)*jdir*dj) 1200 xlon1d = llon + (np.arange(self.nx)*idir*di) 1201 xlons, xlats = np.meshgrid(xlon1d,xlat1d) 1202 rot2ll_vectorized = np.vectorize(arakawa_rotated_grid.rot2ll) 1203 lats, lons = rot2ll_vectorized(xlats,xlons,lasp,losp) 1204 if do_180: 1205 lons = np.where(lons>180.0,lons-360.0,lons) 1206 vector_rotation_angles_vectorized = np.vectorize(arakawa_rotated_grid.vector_rotation_angles) 1207 rots = vector_rotation_angles_vectorized(lats, lons, clat, losp, xlats) 1208 del xlat1d, xlon1d, xlats, xlons 1209 else: 1210 raise ValueError('Unsupported grid') 1211 1212 _latlon_datastore[self._sha1_section3] = dict(latitude=lats,longitude=lons) 1213 try: 1214 _latlon_datastore[self._sha1_section3]['vector_rotation_angles'] = rots 1215 except(NameError): 1216 pass 1217 1218 return lats, lons
Return lats,lons (in degrees) of grid.
Currently can handle reg. lat/lon,cglobal Gaussian, mercator, stereographic, lambert conformal, albers equal-area, space-view and azimuthal equidistant grids.
Parameters
- unrotate: If
True
[DEFAULT], and grid is rotated lat/lon, then unrotate the grid, otherwiseFalse
, do not.
Returns
- lats, lons (numpy.ndarray): Returns two numpy.ndarrays with dtype=numpy.float32 of grid latitudes and longitudes in units of degrees.
1221 def map_keys(self): 1222 """ 1223 Unpack data grid replacing integer values with strings. 1224 1225 These types of fields are categorical or classifications where data 1226 values do not represent an observable or predictable physical quantity. 1227 An example of such a field would be [Dominant Precipitation Type - 1228 DPTYPE](https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table4-201.shtml) 1229 1230 Returns 1231 ------- 1232 map_keys 1233 numpy.ndarray of string values per element. 1234 """ 1235 hold_auto_nans = _AUTO_NANS 1236 set_auto_nans(False) 1237 1238 if (np.all(self.section1[0:2] == [7, 14]) and self.shortName == "PWTHER") or ( 1239 self._isNDFD and self.shortName in {"WX", "WWA"} 1240 ): 1241 keys = utils.decode_wx_strings(self.section2) 1242 if hasattr(self,'priMissingValue') and self.priMissingValue not in [None,0]: 1243 keys[int(self.priMissingValue)] = 'Missing' 1244 if hasattr(self,'secMissingValue') and self.secMissingValue not in [None,0]: 1245 keys[int(self.secMissingValue)] = 'Missing' 1246 u,inv = np.unique(self.data,return_inverse=True) 1247 fld = np.array([keys[x] for x in u])[inv].reshape(self.data.shape) 1248 else: 1249 # For data whose units are defined in a code table (i.e. classification or mask) 1250 tblname = re.findall(r'\d\.\d+',self.units,re.IGNORECASE)[0] 1251 fld = self.data.astype(np.int32).astype(str) 1252 tbl = tables.get_table(tblname,expand=True) 1253 for val in np.unique(fld): 1254 fld = np.where(fld==val,tbl[val],fld) 1255 set_auto_nans(hold_auto_nans) 1256 return fld
Unpack data grid replacing integer values with strings.
These types of fields are categorical or classifications where data values do not represent an observable or predictable physical quantity. An example of such a field would be Dominant Precipitation Type - DPTYPE
Returns
- map_keys: numpy.ndarray of string values per element.
1259 def to_bytes(self, validate: bool=True): 1260 """ 1261 Return packed GRIB2 message in bytes format. 1262 1263 This will be useful for exporting data in non-file formats. For example, 1264 can be used to output grib data directly to S3 using the boto3 client 1265 without the need to write a temporary file to upload first. 1266 1267 Parameters 1268 ---------- 1269 validate: default=True 1270 If `True` (DEFAULT), validates first/last four bytes for proper 1271 formatting, else returns None. If `False`, message is output as is. 1272 1273 Returns 1274 ------- 1275 to_bytes 1276 Returns GRIB2 formatted message as bytes. 1277 """ 1278 if hasattr(self,'_msg'): 1279 if validate: 1280 if self.validate(): 1281 return self._msg 1282 else: 1283 return None 1284 else: 1285 return self._msg 1286 else: 1287 return None
Return packed GRIB2 message in bytes format.
This will be useful for exporting data in non-file formats. For example, can be used to output grib data directly to S3 using the boto3 client without the need to write a temporary file to upload first.
Parameters
- validate (default=True):
If
True
(DEFAULT), validates first/last four bytes for proper formatting, else returns None. IfFalse
, message is output as is.
Returns
- to_bytes: Returns GRIB2 formatted message as bytes.
1290 def interpolate(self, method, grid_def_out, method_options=None, drtn=None, 1291 num_threads=1): 1292 """ 1293 Grib2Message Interpolator 1294 1295 Performs spatial interpolation via the [NCEPLIBS-ip 1296 library](https://github.com/NOAA-EMC/NCEPLIBS-ip). This interpolate 1297 method only supports scalar interpolation. If you need to perform 1298 vector interpolation, use the module-level `grib2io.interpolate` 1299 function. 1300 1301 Parameters 1302 ---------- 1303 method 1304 Interpolate method to use. This can either be an integer or string 1305 using the following mapping: 1306 1307 | Interpolate Scheme | Integer Value | 1308 | :---: | :---: | 1309 | 'bilinear' | 0 | 1310 | 'bicubic' | 1 | 1311 | 'neighbor' | 2 | 1312 | 'budget' | 3 | 1313 | 'spectral' | 4 | 1314 | 'neighbor-budget' | 6 | 1315 1316 grid_def_out : grib2io.Grib2GridDef 1317 Grib2GridDef object of the output grid. 1318 method_options : list of ints, optional 1319 Interpolation options. See the NCEPLIBS-ip documentation for 1320 more information on how these are used. 1321 drtn 1322 Data Representation Template to be used for the returned 1323 interpolated GRIB2 message. When `None`, the data representation 1324 template of the source GRIB2 message is used. Once again, it is the 1325 user's responsibility to properly set the Data Representation 1326 Template attributes. 1327 num_threads : int, optional 1328 Number of OpenMP threads to use for interpolation. The default 1329 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1330 was not built with OpenMP, then this keyword argument and value 1331 will have no impact. 1332 1333 Returns 1334 ------- 1335 interpolate 1336 If interpolating to a grid, a new Grib2Message object is returned. 1337 The GRIB2 metadata of the new Grib2Message object is identical to 1338 the input except where required to be different because of the new 1339 grid specs and possibly a new data representation template. 1340 1341 If interpolating to station points, the interpolated data values are 1342 returned as a numpy.ndarray. 1343 """ 1344 section0 = self.section0 1345 section0[-1] = 0 1346 gds = [0, grid_def_out.npoints, 0, 255, grid_def_out.gdtn] 1347 section3 = np.concatenate((gds,grid_def_out.gdt)) 1348 drtn = self.drtn if drtn is None else drtn 1349 1350 msg = Grib2Message(section0,self.section1,self.section2,section3, 1351 self.section4,None,self.bitMapFlag.value,drtn=drtn) 1352 1353 msg._msgnum = -1 1354 msg._deflist = self._deflist 1355 msg._coordlist = self._coordlist 1356 shape = (msg.ny,msg.nx) 1357 ndim = 2 1358 if msg.typeOfValues == 0: 1359 dtype = 'float32' 1360 elif msg.typeOfValues == 1: 1361 dtype = 'int32' 1362 msg._data = interpolate(self.data,method,Grib2GridDef.from_section3(self.section3),grid_def_out, 1363 method_options=method_options,num_threads=num_threads).reshape(msg.ny,msg.nx) 1364 msg.section5[0] = grid_def_out.npoints 1365 return msg
Grib2Message Interpolator
Performs spatial interpolation via the NCEPLIBS-ip
library. This interpolate
method only supports scalar interpolation. If you need to perform
vector interpolation, use the module-level grib2io.interpolate
function.
Parameters
- method: Interpolate method to use. This can either be an integer or string using the following mapping:
Interpolate Scheme | Integer Value |
---|---|
'bilinear' | 0 |
'bicubic' | 1 |
'neighbor' | 2 |
'budget' | 3 |
'spectral' | 4 |
'neighbor-budget' | 6 |
- grid_def_out (grib2io.Grib2GridDef): Grib2GridDef object of the output grid.
- method_options (list of ints, optional): Interpolation options. See the NCEPLIBS-ip documentation for more information on how these are used.
- drtn: Data Representation Template to be used for the returned
interpolated GRIB2 message. When
None
, the data representation template of the source GRIB2 message is used. Once again, it is the user's responsibility to properly set the Data Representation Template attributes. - num_threads (int, optional): Number of OpenMP threads to use for interpolation. The default value is 1. If NCEPLIBS-ip and grib2io's iplib extension module was not built with OpenMP, then this keyword argument and value will have no impact.
Returns
- interpolate: If interpolating to a grid, a new Grib2Message object is returned. The GRIB2 metadata of the new Grib2Message object is identical to the input except where required to be different because of the new grid specs and possibly a new data representation template.
If interpolating to station points, the interpolated data values are returned as a numpy.ndarray.
1368 def subset(self, lats, lons): 1369 """ 1370 Return a spatial subset. 1371 1372 Currently only supports regular grids of the following types: 1373 1374 | Grid Type | gdtn | 1375 | :---: | :---: | 1376 | Latitude/Longitude, Equidistant Cylindrical, or Plate Carree | 0 | 1377 | Rotated Latitude/Longitude | 1 | 1378 | Mercator | 10 | 1379 | Polar Stereographic | 20 | 1380 | Lambert Conformal | 30 | 1381 | Albers Equal-Area | 31 | 1382 | Gaussian Latitude/Longitude | 40 | 1383 | Equatorial Azimuthal Equidistant Projection | 110 | 1384 1385 Parameters 1386 ---------- 1387 lats 1388 List or tuple of latitudes. The minimum and maximum latitudes will 1389 be used to define the southern and northern boundaries. 1390 1391 The order of the latitudes is not important. The function will 1392 determine which is the minimum and maximum. 1393 1394 The latitudes should be in decimal degrees with 0.0 at the equator, 1395 positive values in the northern hemisphere increasing to 90, and 1396 negative values in the southern hemisphere decreasing to -90. 1397 lons 1398 List or tuple of longitudes. The minimum and maximum longitudes 1399 will be used to define the western and eastern boundaries. 1400 1401 The order of the longitudes is not important. The function will 1402 determine which is the minimum and maximum. 1403 1404 GRIB2 longitudes should be in decimal degrees with 0.0 at the prime 1405 meridian, positive values increasing eastward to 360. There are no 1406 negative GRIB2 longitudes. 1407 1408 The typical west longitudes that start at 0.0 at the prime meridian 1409 and decrease to -180 westward, are converted to GRIB2 longitudes by 1410 '360 - (absolute value of the west longitude)' where typical 1411 eastern longitudes are unchanged as GRIB2 longitudes. 1412 1413 Returns 1414 ------- 1415 subset 1416 A spatial subset of a GRIB2 message. 1417 """ 1418 if self.gdtn not in [0, 1, 10, 20, 30, 31, 40, 110]: 1419 raise ValueError( 1420 """ 1421 1422Subset only works for 1423 Latitude/Longitude, Equidistant Cylindrical, or Plate Carree (gdtn=0) 1424 Rotated Latitude/Longitude (gdtn=1) 1425 Mercator (gdtn=10) 1426 Polar Stereographic (gdtn=20) 1427 Lambert Conformal (gdtn=30) 1428 Albers Equal-Area (gdtn=31) 1429 Gaussian Latitude/Longitude (gdtn=40) 1430 Equatorial Azimuthal Equidistant Projection (gdtn=110) 1431 1432""" 1433 ) 1434 1435 if self.nx == 0 or self.ny == 0: 1436 raise ValueError( 1437 """ 1438 1439Subset only works for regular grids. 1440 1441""" 1442 ) 1443 1444 newmsg = Grib2Message( 1445 np.copy(self.section0), 1446 np.copy(self.section1), 1447 np.copy(self.section2), 1448 np.copy(self.section3), 1449 np.copy(self.section4), 1450 np.copy(self.section5), 1451 ) 1452 1453 msglats, msglons = self.grid() 1454 1455 la1 = np.max(lats) 1456 lo1 = np.min(lons) 1457 la2 = np.min(lats) 1458 lo2 = np.max(lons) 1459 1460 # Find the indices of the first and last grid points to the nearest 1461 # lat/lon values in the grid. 1462 first_lat = np.abs(msglats - la1) 1463 first_lon = np.abs(msglons - lo1) 1464 max_idx = np.maximum(first_lat, first_lon) 1465 first_j, first_i = np.where(max_idx == np.min(max_idx)) 1466 1467 last_lat = np.abs(msglats - la2) 1468 last_lon = np.abs(msglons - lo2) 1469 max_idx = np.maximum(last_lat, last_lon) 1470 last_j, last_i = np.where(max_idx == np.min(max_idx)) 1471 1472 setattr(newmsg, "latitudeFirstGridpoint", msglats[first_j[0], first_i[0]]) 1473 setattr(newmsg, "longitudeFirstGridpoint", msglons[first_j[0], first_i[0]]) 1474 setattr(newmsg, "nx", np.abs(first_i[0] - last_i[0])) 1475 setattr(newmsg, "ny", np.abs(first_j[0] - last_j[0])) 1476 1477 # Set *LastGridpoint attributes even if only used for gdtn=[0, 1, 40]. 1478 # This information is used to subset xarray datasets and even though 1479 # unnecessary for some supported grid types, it won't affect a grib2io 1480 # message to set them. 1481 setattr(newmsg, "latitudeLastGridpoint", msglats[last_j[0], last_i[0]]) 1482 setattr(newmsg, "longitudeLastGridpoint", msglons[last_j[0], last_i[0]]) 1483 1484 setattr( 1485 newmsg, 1486 "data", 1487 self.data[ 1488 min(first_j[0], last_j[0]) : max(first_j[0], last_j[0]), 1489 min(first_i[0], last_i[0]) : max(first_i[0], last_i[0]), 1490 ].copy(), 1491 ) 1492 1493 # Need to set the newmsg._sha1_section3 to a blank string so the grid 1494 # method ignores the cached lat/lon values. This will force the grid 1495 # method to recompute the lat/lon values for the subsetted grid. 1496 newmsg._sha1_section3 = "" 1497 newmsg.grid() 1498 1499 return newmsg
Return a spatial subset.
Currently only supports regular grids of the following types:
Grid Type | gdtn |
---|---|
Latitude/Longitude, Equidistant Cylindrical, or Plate Carree | 0 |
Rotated Latitude/Longitude | 1 |
Mercator | 10 |
Polar Stereographic | 20 |
Lambert Conformal | 30 |
Albers Equal-Area | 31 |
Gaussian Latitude/Longitude | 40 |
Equatorial Azimuthal Equidistant Projection | 110 |
Parameters
- lats: List or tuple of latitudes. The minimum and maximum latitudes will be used to define the southern and northern boundaries.
The order of the latitudes is not important. The function will determine which is the minimum and maximum.
The latitudes should be in decimal degrees with 0.0 at the equator, positive values in the northern hemisphere increasing to 90, and negative values in the southern hemisphere decreasing to -90.
- lons: List or tuple of longitudes. The minimum and maximum longitudes will be used to define the western and eastern boundaries.
The order of the longitudes is not important. The function will determine which is the minimum and maximum.
GRIB2 longitudes should be in decimal degrees with 0.0 at the prime meridian, positive values increasing eastward to 360. There are no negative GRIB2 longitudes.
The typical west longitudes that start at 0.0 at the prime meridian and decrease to -180 westward, are converted to GRIB2 longitudes by '360 - (absolute value of the west longitude)' where typical eastern longitudes are unchanged as GRIB2 longitudes.
Returns
- subset: A spatial subset of a GRIB2 message.
1501 def validate(self): 1502 """ 1503 Validate a complete GRIB2 message. 1504 1505 The g2c library does its own internal validation when g2_gribend() is called, but 1506 we will check in grib2io also. The validation checks if the first 4 bytes in 1507 self._msg is 'GRIB' and '7777' as the last 4 bytes and that the message length in 1508 section 0 equals the length of the packed message. 1509 1510 Returns 1511 ------- 1512 `True` if the packed GRIB2 message is complete and well-formed, `False` otherwise. 1513 """ 1514 valid = False 1515 if hasattr(self,'_msg'): 1516 if self._msg[0:4]+self._msg[-4:] == b'GRIB7777': 1517 if self.section0[-1] == len(self._msg): 1518 valid = True 1519 return valid
Validate a complete GRIB2 message.
The g2c library does its own internal validation when g2_gribend() is called, but we will check in grib2io also. The validation checks if the first 4 bytes in self._msg is 'GRIB' and '7777' as the last 4 bytes and that the message length in section 0 equals the length of the packed message.
Returns
True
if the packed GRIB2 message is complete and well-formed,False
otherwise.
1992@dataclass 1993class Grib2GridDef: 1994 """ 1995 Class for Grid Definition Template Number and Template as attributes. 1996 1997 This allows for cleaner looking code when passing these metadata around. 1998 For example, the `grib2io._Grib2Message.interpolate` method and 1999 `grib2io.interpolate` function accepts these objects. 2000 """ 2001 gdtn: int 2002 gdt: NDArray 2003 2004 @classmethod 2005 def from_section3(cls, section3): 2006 return cls(section3[4],section3[5:]) 2007 2008 @property 2009 def nx(self): 2010 """Number of grid points in x-direction.""" 2011 return self.gdt[7] 2012 2013 @property 2014 def ny(self): 2015 """Number of grid points in y-direction.""" 2016 return self.gdt[8] 2017 2018 @property 2019 def npoints(self): 2020 """Total number of grid points.""" 2021 return self.gdt[7] * self.gdt[8] 2022 2023 @property 2024 def shape(self): 2025 """Shape of the grid.""" 2026 return (int(self.ny), int(self.nx))
Class for Grid Definition Template Number and Template as attributes.
This allows for cleaner looking code when passing these metadata around.
For example, the grib2io._Grib2Message.interpolate
method and
grib2io.interpolate
function accepts these objects.
2008 @property 2009 def nx(self): 2010 """Number of grid points in x-direction.""" 2011 return self.gdt[7]
Number of grid points in x-direction.
2013 @property 2014 def ny(self): 2015 """Number of grid points in y-direction.""" 2016 return self.gdt[8]
Number of grid points in y-direction.