grib2io
Introduction
grib2io is a Python package that provides an interface to the NCEP GRIB2 C (g2c) library for the purpose of reading and writing WMO GRIdded Binary, Edition 2 (GRIB2) messages. A physical file can contain one or more GRIB2 messages.
GRIB2 file IO is performed directly in Python. The unpacking/packing of GRIB2 integer, coded metadata and data sections is performed by the g2c library functions via the g2clib Cython extension module. The decoding/encoding of GRIB2 metadata is translated into more descriptive, plain language metadata by providing a mapping of the integer coded metadata to the appropriate GRIB2 code tables. The NCEP GRIB2 code tables are a part of the grib2io package.
Interpolation
As of grib2io v2.4.0, spatial interpolation via NCEPLIPS-ip Fortran library is now a part of the grib2io package. The separate component package, grib2io-interp, has been deprecated. grib2io-interp provided interpolation via F2PY interface to NCEPLIBS-ip, which has become difficult since the removal of distutils from Python 3.12+.
NCEPLIBS-ip interpolation Fortran subroutines contain the BIND(C)
attribute which
provides an equivalent C-interface. grib2io now provides a Cython-based interface, iplib
,
to these Fortran subroutines via their C-interface. If NCEPLIBS-ip was built with
OpenMP support, iplib
will provide functions for getting and setting the number of
OpenMP threads.
Tutorials
The following Jupyter Notebooks are available as tutorials:
1from ._grib2io import * 2from ._grib2io import __doc__ 3from ._grib2io import _Grib2Message 4 5__all__ = ['open', 'show_config', 'interpolate', 'interpolate_to_stations', 6 'tables', 'templates', 'utils', 7 'Grib2Message', '_Grib2Message', 'Grib2GridDef'] 8 9try: 10 from . import __config__ 11 __version__ = __config__.grib2io_version 12 has_interpolation = __config__.has_interpolation 13 has_openmp_support = __config__.has_openmp_support 14 g2c_static = __config__.g2c_static 15 ip_static = __config__.ip_static 16 extra_objects = __config__.extra_objects 17except(ImportError): 18 pass 19 20from .g2clib import __version__ as __g2clib_version__ 21from .g2clib import _has_jpeg 22from .g2clib import _has_png 23from .g2clib import _has_aec 24 25has_jpeg_support = bool(_has_jpeg) 26has_png_support = bool(_has_png) 27has_aec_support = bool(_has_aec) 28 29from .tables.originating_centers import _ncep_grib2_table_version 30ncep_grib2_table_version = _ncep_grib2_table_version 31g2c_version = __g2clib_version__ 32 33def show_config(): 34 """Print grib2io build configuration information.""" 35 print(f'grib2io version {__version__} Configuration:') 36 print(f'') 37 print(f'NCEPLIBS-g2c library version: {__g2clib_version__}') 38 print(f'\tStatic library: {g2c_static}') 39 print(f'\tJPEG compression support: {has_jpeg_support}') 40 print(f'\tPNG compression support: {has_png_support}') 41 print(f'\tAEC compression support: {has_aec_support}') 42 print(f'') 43 print(f'NCEPLIPS-ip support: {has_interpolation}') 44 print(f'\tStatic library: {ip_static}') 45 print(f'\tOpenMP support: {has_openmp_support}') 46 print(f'') 47 print(f'Static libs:') 48 for lib in extra_objects: 49 print(f'\t{lib}') 50 print(f'') 51 print(f'NCEP GRIB2 Table Version: {_ncep_grib2_table_version}')
86class open(): 87 """ 88 GRIB2 File Object. 89 90 A physical file can contain one or more GRIB2 messages. When instantiated, 91 class `grib2io.open`, the file named `filename` is opened for reading (`mode 92 = 'r'`) and is automatically indexed. The indexing procedure reads some of 93 the GRIB2 metadata for all GRIB2 Messages. A GRIB2 Message may contain 94 submessages whereby Section 2-7 can be repeated. grib2io accommodates for 95 this by flattening any GRIB2 submessages into multiple individual messages. 96 97 It is important to note that GRIB2 files from some Meteorological agencies 98 contain other data than GRIB2 messages. GRIB2 files from ECMWF can contain 99 GRIB1 and GRIB2 messages. grib2io checks for these and safely ignores them. 100 101 Attributes 102 ---------- 103 closed : bool 104 `True` is file handle is close; `False` otherwise. 105 current_message : int 106 Current position of the file in units of GRIB2 Messages. 107 levels : tuple 108 Tuple containing a unique list of wgrib2-formatted level/layer strings. 109 messages : int 110 Count of GRIB2 Messages contained in the file. 111 mode : str 112 File IO mode of opening the file. 113 name : str 114 Full path name of the GRIB2 file. 115 size : int 116 Size of the file in units of bytes. 117 variables : tuple 118 Tuple containing a unique list of variable short names (i.e. GRIB2 119 abbreviation names). 120 """ 121 122 __slots__ = ('_fileid', '_filehandle', '_hasindex', '_index', '_nodata', 123 '_pos', 'closed', 'current_message', 'messages', 'mode', 124 'name', 'size') 125 126 def __init__( 127 self, 128 filename: Union[bytes, str, Path], 129 mode: Literal["r", "w", "x"] = "r", 130 **kwargs, 131 ): 132 """ 133 Initialize GRIB2 File object instance. 134 135 Parameters 136 ---------- 137 filename 138 File name containing GRIB2 messages. 139 OR 140 bytes 141 mode: default="r" 142 File access mode where "r" opens the files for reading only; "w" 143 opens the file for overwriting and "x" for writing to a new file. 144 """ 145 146 # Manage keywords 147 if "_xarray_backend" not in kwargs: 148 kwargs["_xarray_backend"] = False 149 self._nodata = False 150 else: 151 self._nodata = kwargs["_xarray_backend"] 152 153 # All write modes are read/write. 154 # All modes are binary. 155 if mode in ("a", "x", "w"): 156 mode += "+" 157 mode = mode + "b" 158 159 if isinstance(filename, bytes): 160 if filename[:2] == _GZIP_HEADER: 161 filename = gzip.decompress(filename) 162 if filename[:4]+filename[-4:] != b'GRIB7777': 163 raise ValueError("Invalid GRIB bytes") 164 self._filehandle = BytesIO(filename) 165 self.name = "<in-memory-file>" 166 self.size = len(filename) 167 self._fileid = hashlib.sha1((self.name+str(self.size)).encode('ASCII')).hexdigest() 168 169 # Some GRIB2 files are gzipped, so check for that here, but 170 # raise error when using xarray backend. 171 else: 172 if 'r' in mode: 173 self._filehandle = builtins.open(filename, mode=mode) 174 # Gzip files contain a 2-byte header b'\x1f\x8b'. 175 if self._filehandle.read(2) == _GZIP_HEADER: 176 self._filehandle.close() 177 if kwargs["_xarray_backend"]: 178 raise RuntimeError('Gzip GRIB2 files are not supported by the Xarray backend.') 179 self._filehandle = gzip.open(filename, mode=mode) 180 else: 181 self._filehandle = builtins.open(filename, mode=mode) 182 else: 183 self._filehandle = builtins.open(filename, mode=mode) 184 self.name = os.path.abspath(filename) 185 self.name = os.path.abspath(filename) 186 fstat = os.stat(self.name) 187 self.size = fstat.st_size 188 self._fileid = hashlib.sha1((self.name+str(fstat.st_ino)+ 189 str(self.size)).encode('ASCII')).hexdigest() 190 191 self._hasindex = False 192 self._index = {} 193 self.mode = mode 194 self.messages = 0 195 self.current_message = 0 196 self.closed = self._filehandle.closed 197 if 'r' in self.mode: 198 self._build_index() 199 # FIX: Cannot perform reads on mode='a' 200 #if 'a' in self.mode and self.size > 0: self._build_index() 201 202 203 def __delete__(self, instance): 204 self.close() 205 del self._index 206 207 208 def __enter__(self): 209 return self 210 211 212 def __exit__(self, atype, value, traceback): 213 self.close() 214 215 216 def __iter__(self): 217 yield from self._index['msg'] 218 219 220 def __len__(self): 221 return self.messages 222 223 224 def __repr__(self): 225 strings = [] 226 for k in self.__slots__: 227 if k.startswith('_'): continue 228 strings.append('%s = %s\n'%(k,eval('self.'+k))) 229 return ''.join(strings) 230 231 232 def __getitem__(self, key): 233 if isinstance(key,int): 234 if abs(key) >= len(self._index['msg']): 235 raise IndexError("index out of range") 236 else: 237 return self._index['msg'][key] 238 elif isinstance(key,str): 239 return self.select(shortName=key) 240 elif isinstance(key,slice): 241 return self._index['msg'][key] 242 else: 243 raise KeyError('Key must be an integer, slice, or GRIB2 variable shortName.') 244 245 246 def _build_index(self): 247 """Perform indexing of GRIB2 Messages.""" 248 # Initialize index dictionary 249 if not self._hasindex: 250 self._index['sectionOffset'] = [] 251 self._index['sectionSize'] = [] 252 self._index['msgSize'] = [] 253 self._index['msgNumber'] = [] 254 self._index['msg'] = [] 255 self._index['isSubmessage'] = [] 256 self._hasindex = True 257 258 # Iterate 259 while True: 260 try: 261 # Initialize 262 bmapflag = None 263 pos = self._filehandle.tell() 264 section2 = b'' 265 trailer = b'' 266 _secpos = dict.fromkeys(range(8)) 267 _secsize = dict.fromkeys(range(8)) 268 _isSubmessage = False 269 270 # Ignore headers (usually text) that are not part of the GRIB2 271 # file. For example, NAVGEM files have a http header at the 272 # beginning that needs to be ignored. 273 274 # Read a byte at a time until "GRIB" is found. Using 275 # "wgrib2" on a NAVGEM file, the header was 421 bytes and 276 # decided to go to 2048 bytes to be safe. For normal GRIB2 277 # files this should be quick and break out of the first 278 # loop when "test_offset" is 0. 279 for test_offset in range(2048): 280 self._filehandle.seek(pos + test_offset) 281 header = struct.unpack(">I", self._filehandle.read(4))[0] 282 if header.to_bytes(4, "big") == b"GRIB": 283 pos = pos + test_offset 284 break 285 else: 286 # NOTE: Coming here means that no "GRIB" message identifier 287 # was found in the previous 2048 bytes. So here we continue 288 # the while True loop. 289 continue 290 291 # Read the rest of Section 0 using struct. 292 _secpos[0] = self._filehandle.tell()-4 293 _secsize[0] = 16 294 secmsg = self._filehandle.read(12) 295 section0 = np.concatenate(([header],list(struct.unpack('>HBBQ',secmsg))),dtype=np.int64) 296 297 # Make sure message is GRIB2. 298 if section0[3] != 2: 299 # Check for GRIB1 and ignore. 300 if secmsg[3] == 1: 301 warnings.warn("GRIB version 1 message detected. Ignoring...") 302 grib1_size = int.from_bytes(secmsg[:3],"big") 303 self._filehandle.seek(self._filehandle.tell()+grib1_size-16) 304 continue 305 else: 306 raise ValueError("Bad GRIB version number.") 307 308 # Read and unpack sections 1 through 8 which all follow a 309 # pattern of section size, number, and content. 310 while 1: 311 # Read first 5 bytes of the section which contains the size 312 # of the section (4 bytes) and the section number (1 byte). 313 secmsg = self._filehandle.read(5) 314 secsize, secnum = struct.unpack('>iB',secmsg) 315 316 # Record the offset of the section number and "append" the 317 # rest of the section to secmsg. 318 _secpos[secnum] = self._filehandle.tell()-5 319 _secsize[secnum] = secsize 320 if secnum in {1,3,4,5}: 321 secmsg += self._filehandle.read(secsize-5) 322 grbpos = 0 323 324 # Unpack section 325 if secnum == 1: 326 # Unpack Section 1 327 section1, grbpos = g2clib.unpack1(secmsg) 328 elif secnum == 2: 329 # Unpack Section 2 330 section2 = self._filehandle.read(secsize-5) 331 elif secnum == 3: 332 # Unpack Section 3 333 gds, gdt, deflist, grbpos = g2clib.unpack3(secmsg) 334 gds = gds.tolist() 335 gdt = gdt.tolist() 336 section3 = np.concatenate((gds,gdt)) 337 section3 = np.where(section3==4294967295,-1,section3) 338 elif secnum == 4: 339 # Unpack Section 4 340 pdtnum, pdt, coordlist, numcoord, grbpos = g2clib.unpack4(secmsg) 341 pdt = pdt.tolist() 342 section4 = np.concatenate((np.array((numcoord,pdtnum)),pdt)) 343 elif secnum == 5: 344 # Unpack Section 5 345 drtn, drt, npts, self._pos = g2clib.unpack5(secmsg) 346 section5 = np.concatenate((np.array((npts,drtn)),drt)) 347 section5 = np.where(section5==4294967295,-1,section5) 348 elif secnum == 6: 349 # Unpack Section 6 - Just the bitmap flag 350 bmapflag = struct.unpack('>B',self._filehandle.read(1))[0] 351 if bmapflag == 0: 352 bmappos = self._filehandle.tell()-6 353 elif bmapflag == 254: 354 # Do this to keep the previous position value 355 pass 356 else: 357 bmappos = None 358 self._filehandle.seek(self._filehandle.tell()+secsize-6) 359 elif secnum == 7: 360 # Do not unpack section 7 here, but move the file pointer 361 # to after section 7. 362 self._filehandle.seek(self._filehandle.tell()+secsize-5) 363 364 # Update the file index. 365 self.messages += 1 366 self._index['sectionOffset'].append(copy.deepcopy(_secpos)) 367 self._index['sectionSize'].append(copy.deepcopy(_secsize)) 368 self._index['msgSize'].append(section0[-1]) 369 self._index['msgNumber'].append(self.messages) 370 self._index['isSubmessage'].append(_isSubmessage) 371 372 # Create Grib2Message with data. 373 msg = Grib2Message(section0,section1,section2,section3,section4,section5,bmapflag) 374 msg._msgnum = self.messages-1 375 msg._deflist = deflist 376 msg._coordlist = coordlist 377 if not self._nodata: 378 msg._ondiskarray = Grib2MessageOnDiskArray((msg.ny,msg.nx), 2, 379 TYPE_OF_VALUES_DTYPE[msg.typeOfValues], 380 self._filehandle, 381 msg, pos, _secpos[6], _secpos[7]) 382 self._index['msg'].append(msg) 383 384 # If here, then we have moved through GRIB2 section 1-7. 385 # Now we need to check the next 4 bytes after section 7. 386 trailer = struct.unpack('>i',self._filehandle.read(4))[0] 387 388 # If we reach the GRIB2 trailer string ('7777'), then we 389 # can break begin processing the next GRIB2 message. If 390 # not, then we continue within the same iteration to 391 # process a GRIB2 submessage. 392 if trailer.to_bytes(4, "big") == b'7777': 393 break 394 else: 395 # If here, trailer should be the size of the first 396 # section of the next submessage, then the next byte 397 # is the section number. Check this value. 398 nextsec = struct.unpack('>B',self._filehandle.read(1))[0] 399 if nextsec not in {2,3,4}: 400 raise ValueError("Bad GRIB2 message structure.") 401 self._filehandle.seek(self._filehandle.tell()-5) 402 _isSubmessage = True 403 continue 404 else: 405 raise ValueError("Bad GRIB2 section number.") 406 407 except(struct.error): 408 if 'r' in self.mode: 409 self._filehandle.seek(0) 410 break 411 412 413 @property 414 def levels(self): 415 """Provides a unique tuple of level strings.""" 416 if self._hasindex and not self._nodata: 417 return tuple(sorted(set([msg.level for msg in self._index['msg']]))) 418 else: 419 return None 420 421 422 @property 423 def variables(self): 424 """Provides a unique tuple of variable shortName strings.""" 425 if self._hasindex and not self._nodata: 426 return tuple(sorted(set([msg.shortName for msg in self._index['msg']]))) 427 else: 428 return None 429 430 431 def close(self): 432 """Close the file handle.""" 433 if not self._filehandle.closed: 434 self.messages = 0 435 self.current_message = 0 436 self._filehandle.close() 437 self.closed = self._filehandle.closed 438 439 440 def read(self, size: Optional[int]=None): 441 """ 442 Read size amount of GRIB2 messages from the current position. 443 444 If no argument is given, then size is None and all messages are returned 445 from the current position in the file. This read method follows the 446 behavior of Python's builtin open() function, but whereas that operates 447 on units of bytes, we operate on units of GRIB2 messages. 448 449 Parameters 450 ---------- 451 size: default=None 452 The number of GRIB2 messages to read from the current position. If 453 no argument is give, the default value is None and remainder of 454 the file is read. 455 456 Returns 457 ------- 458 read 459 ``Grib2Message`` object when size = 1 or a list of Grib2Messages 460 when size > 1. 461 """ 462 if size is not None and size < 0: 463 size = None 464 if size is None or size > 1: 465 start = self.tell() 466 stop = self.messages if size is None else start+size 467 if size is None: 468 self.current_message = self.messages-1 469 else: 470 self.current_message += size 471 return self._index['msg'][slice(start,stop,1)] 472 elif size == 1: 473 self.current_message += 1 474 return self._index['msg'][self.current_message] 475 else: 476 None 477 478 479 def seek(self, pos: int): 480 """ 481 Set the position within the file in units of GRIB2 messages. 482 483 Parameters 484 ---------- 485 pos 486 The GRIB2 Message number to set the file pointer to. 487 """ 488 if self._hasindex: 489 self._filehandle.seek(self._index['sectionOffset'][0][pos]) 490 self.current_message = pos 491 492 493 def tell(self): 494 """Returns the position of the file in units of GRIB2 Messages.""" 495 return self.current_message 496 497 498 def select(self, **kwargs): 499 """Select GRIB2 messages by `Grib2Message` attributes.""" 500 # TODO: Added ability to process multiple values for each keyword (attribute) 501 idxs = [] 502 nkeys = len(kwargs.keys()) 503 for k,v in kwargs.items(): 504 for m in self._index['msg']: 505 if hasattr(m,k) and getattr(m,k) == v: idxs.append(m._msgnum) 506 idxs = np.array(idxs,dtype=np.int32) 507 return [self._index['msg'][i] for i in [ii[0] for ii in collections.Counter(idxs).most_common() if ii[1] == nkeys]] 508 509 510 def write(self, msg): 511 """ 512 Writes GRIB2 message object to file. 513 514 Parameters 515 ---------- 516 msg 517 GRIB2 message objects to write to file. 518 """ 519 if isinstance(msg,list): 520 for m in msg: 521 self.write(m) 522 return 523 524 if issubclass(msg.__class__,_Grib2Message): 525 if hasattr(msg,'_msg'): 526 self._filehandle.write(msg._msg) 527 else: 528 if msg._signature != msg._generate_signature(): 529 msg.pack() 530 self._filehandle.write(msg._msg) 531 else: 532 if hasattr(msg._data,'filehandle'): 533 msg._data.filehandle.seek(msg._data.offset) 534 self._filehandle.write(msg._data.filehandle.read(msg.section0[-1])) 535 else: 536 msg.pack() 537 self._filehandle.write(msg._msg) 538 self.flush() 539 self.size = os.path.getsize(self.name) 540 self._filehandle.seek(self.size-msg.section0[-1]) 541 self._build_index() 542 else: 543 raise TypeError("msg must be a Grib2Message object.") 544 return 545 546 547 def flush(self): 548 """Flush the file object buffer.""" 549 self._filehandle.flush() 550 551 552 def levels_by_var(self, name: str): 553 """ 554 Return a list of level strings given a variable shortName. 555 556 Parameters 557 ---------- 558 name 559 Grib2Message variable shortName 560 561 Returns 562 ------- 563 levels_by_var 564 A list of unique level strings. 565 """ 566 return list(sorted(set([msg.level for msg in self.select(shortName=name)]))) 567 568 569 def vars_by_level(self, level: str): 570 """ 571 Return a list of variable shortName strings given a level. 572 573 Parameters 574 ---------- 575 level 576 Grib2Message variable level 577 578 Returns 579 ------- 580 vars_by_level 581 A list of unique variable shortName strings. 582 """ 583 return list(sorted(set([msg.shortName for msg in self.select(level=level)])))
GRIB2 File Object.
A physical file can contain one or more GRIB2 messages. When instantiated,
class grib2io.open
, the file named filename
is opened for reading (mode
= 'r'
) and is automatically indexed. The indexing procedure reads some of
the GRIB2 metadata for all GRIB2 Messages. A GRIB2 Message may contain
submessages whereby Section 2-7 can be repeated. grib2io accommodates for
this by flattening any GRIB2 submessages into multiple individual messages.
It is important to note that GRIB2 files from some Meteorological agencies contain other data than GRIB2 messages. GRIB2 files from ECMWF can contain GRIB1 and GRIB2 messages. grib2io checks for these and safely ignores them.
Attributes
- closed (bool):
True
is file handle is close;False
otherwise. - current_message (int): Current position of the file in units of GRIB2 Messages.
- levels (tuple): Tuple containing a unique list of wgrib2-formatted level/layer strings.
- messages (int): Count of GRIB2 Messages contained in the file.
- mode (str): File IO mode of opening the file.
- name (str): Full path name of the GRIB2 file.
- size (int): Size of the file in units of bytes.
- variables (tuple): Tuple containing a unique list of variable short names (i.e. GRIB2 abbreviation names).
126 def __init__( 127 self, 128 filename: Union[bytes, str, Path], 129 mode: Literal["r", "w", "x"] = "r", 130 **kwargs, 131 ): 132 """ 133 Initialize GRIB2 File object instance. 134 135 Parameters 136 ---------- 137 filename 138 File name containing GRIB2 messages. 139 OR 140 bytes 141 mode: default="r" 142 File access mode where "r" opens the files for reading only; "w" 143 opens the file for overwriting and "x" for writing to a new file. 144 """ 145 146 # Manage keywords 147 if "_xarray_backend" not in kwargs: 148 kwargs["_xarray_backend"] = False 149 self._nodata = False 150 else: 151 self._nodata = kwargs["_xarray_backend"] 152 153 # All write modes are read/write. 154 # All modes are binary. 155 if mode in ("a", "x", "w"): 156 mode += "+" 157 mode = mode + "b" 158 159 if isinstance(filename, bytes): 160 if filename[:2] == _GZIP_HEADER: 161 filename = gzip.decompress(filename) 162 if filename[:4]+filename[-4:] != b'GRIB7777': 163 raise ValueError("Invalid GRIB bytes") 164 self._filehandle = BytesIO(filename) 165 self.name = "<in-memory-file>" 166 self.size = len(filename) 167 self._fileid = hashlib.sha1((self.name+str(self.size)).encode('ASCII')).hexdigest() 168 169 # Some GRIB2 files are gzipped, so check for that here, but 170 # raise error when using xarray backend. 171 else: 172 if 'r' in mode: 173 self._filehandle = builtins.open(filename, mode=mode) 174 # Gzip files contain a 2-byte header b'\x1f\x8b'. 175 if self._filehandle.read(2) == _GZIP_HEADER: 176 self._filehandle.close() 177 if kwargs["_xarray_backend"]: 178 raise RuntimeError('Gzip GRIB2 files are not supported by the Xarray backend.') 179 self._filehandle = gzip.open(filename, mode=mode) 180 else: 181 self._filehandle = builtins.open(filename, mode=mode) 182 else: 183 self._filehandle = builtins.open(filename, mode=mode) 184 self.name = os.path.abspath(filename) 185 self.name = os.path.abspath(filename) 186 fstat = os.stat(self.name) 187 self.size = fstat.st_size 188 self._fileid = hashlib.sha1((self.name+str(fstat.st_ino)+ 189 str(self.size)).encode('ASCII')).hexdigest() 190 191 self._hasindex = False 192 self._index = {} 193 self.mode = mode 194 self.messages = 0 195 self.current_message = 0 196 self.closed = self._filehandle.closed 197 if 'r' in self.mode: 198 self._build_index() 199 # FIX: Cannot perform reads on mode='a' 200 #if 'a' in self.mode and self.size > 0: self._build_index()
Initialize GRIB2 File object instance.
Parameters
- filename: File name containing GRIB2 messages. OR bytes
- mode (default="r"): File access mode where "r" opens the files for reading only; "w" opens the file for overwriting and "x" for writing to a new file.
413 @property 414 def levels(self): 415 """Provides a unique tuple of level strings.""" 416 if self._hasindex and not self._nodata: 417 return tuple(sorted(set([msg.level for msg in self._index['msg']]))) 418 else: 419 return None
Provides a unique tuple of level strings.
422 @property 423 def variables(self): 424 """Provides a unique tuple of variable shortName strings.""" 425 if self._hasindex and not self._nodata: 426 return tuple(sorted(set([msg.shortName for msg in self._index['msg']]))) 427 else: 428 return None
Provides a unique tuple of variable shortName strings.
431 def close(self): 432 """Close the file handle.""" 433 if not self._filehandle.closed: 434 self.messages = 0 435 self.current_message = 0 436 self._filehandle.close() 437 self.closed = self._filehandle.closed
Close the file handle.
440 def read(self, size: Optional[int]=None): 441 """ 442 Read size amount of GRIB2 messages from the current position. 443 444 If no argument is given, then size is None and all messages are returned 445 from the current position in the file. This read method follows the 446 behavior of Python's builtin open() function, but whereas that operates 447 on units of bytes, we operate on units of GRIB2 messages. 448 449 Parameters 450 ---------- 451 size: default=None 452 The number of GRIB2 messages to read from the current position. If 453 no argument is give, the default value is None and remainder of 454 the file is read. 455 456 Returns 457 ------- 458 read 459 ``Grib2Message`` object when size = 1 or a list of Grib2Messages 460 when size > 1. 461 """ 462 if size is not None and size < 0: 463 size = None 464 if size is None or size > 1: 465 start = self.tell() 466 stop = self.messages if size is None else start+size 467 if size is None: 468 self.current_message = self.messages-1 469 else: 470 self.current_message += size 471 return self._index['msg'][slice(start,stop,1)] 472 elif size == 1: 473 self.current_message += 1 474 return self._index['msg'][self.current_message] 475 else: 476 None
Read size amount of GRIB2 messages from the current position.
If no argument is given, then size is None and all messages are returned from the current position in the file. This read method follows the behavior of Python's builtin open() function, but whereas that operates on units of bytes, we operate on units of GRIB2 messages.
Parameters
- size (default=None): The number of GRIB2 messages to read from the current position. If no argument is give, the default value is None and remainder of the file is read.
Returns
- read:
Grib2Message
object when size = 1 or a list of Grib2Messages when size > 1.
479 def seek(self, pos: int): 480 """ 481 Set the position within the file in units of GRIB2 messages. 482 483 Parameters 484 ---------- 485 pos 486 The GRIB2 Message number to set the file pointer to. 487 """ 488 if self._hasindex: 489 self._filehandle.seek(self._index['sectionOffset'][0][pos]) 490 self.current_message = pos
Set the position within the file in units of GRIB2 messages.
Parameters
- pos: The GRIB2 Message number to set the file pointer to.
493 def tell(self): 494 """Returns the position of the file in units of GRIB2 Messages.""" 495 return self.current_message
Returns the position of the file in units of GRIB2 Messages.
498 def select(self, **kwargs): 499 """Select GRIB2 messages by `Grib2Message` attributes.""" 500 # TODO: Added ability to process multiple values for each keyword (attribute) 501 idxs = [] 502 nkeys = len(kwargs.keys()) 503 for k,v in kwargs.items(): 504 for m in self._index['msg']: 505 if hasattr(m,k) and getattr(m,k) == v: idxs.append(m._msgnum) 506 idxs = np.array(idxs,dtype=np.int32) 507 return [self._index['msg'][i] for i in [ii[0] for ii in collections.Counter(idxs).most_common() if ii[1] == nkeys]]
Select GRIB2 messages by Grib2Message
attributes.
510 def write(self, msg): 511 """ 512 Writes GRIB2 message object to file. 513 514 Parameters 515 ---------- 516 msg 517 GRIB2 message objects to write to file. 518 """ 519 if isinstance(msg,list): 520 for m in msg: 521 self.write(m) 522 return 523 524 if issubclass(msg.__class__,_Grib2Message): 525 if hasattr(msg,'_msg'): 526 self._filehandle.write(msg._msg) 527 else: 528 if msg._signature != msg._generate_signature(): 529 msg.pack() 530 self._filehandle.write(msg._msg) 531 else: 532 if hasattr(msg._data,'filehandle'): 533 msg._data.filehandle.seek(msg._data.offset) 534 self._filehandle.write(msg._data.filehandle.read(msg.section0[-1])) 535 else: 536 msg.pack() 537 self._filehandle.write(msg._msg) 538 self.flush() 539 self.size = os.path.getsize(self.name) 540 self._filehandle.seek(self.size-msg.section0[-1]) 541 self._build_index() 542 else: 543 raise TypeError("msg must be a Grib2Message object.") 544 return
Writes GRIB2 message object to file.
Parameters
- msg: GRIB2 message objects to write to file.
552 def levels_by_var(self, name: str): 553 """ 554 Return a list of level strings given a variable shortName. 555 556 Parameters 557 ---------- 558 name 559 Grib2Message variable shortName 560 561 Returns 562 ------- 563 levels_by_var 564 A list of unique level strings. 565 """ 566 return list(sorted(set([msg.level for msg in self.select(shortName=name)])))
Return a list of level strings given a variable shortName.
Parameters
- name: Grib2Message variable shortName
Returns
- levels_by_var: A list of unique level strings.
569 def vars_by_level(self, level: str): 570 """ 571 Return a list of variable shortName strings given a level. 572 573 Parameters 574 ---------- 575 level 576 Grib2Message variable level 577 578 Returns 579 ------- 580 vars_by_level 581 A list of unique variable shortName strings. 582 """ 583 return list(sorted(set([msg.shortName for msg in self.select(level=level)])))
Return a list of variable shortName strings given a level.
Parameters
- level: Grib2Message variable level
Returns
- vars_by_level: A list of unique variable shortName strings.
34def show_config(): 35 """Print grib2io build configuration information.""" 36 print(f'grib2io version {__version__} Configuration:') 37 print(f'') 38 print(f'NCEPLIBS-g2c library version: {__g2clib_version__}') 39 print(f'\tStatic library: {g2c_static}') 40 print(f'\tJPEG compression support: {has_jpeg_support}') 41 print(f'\tPNG compression support: {has_png_support}') 42 print(f'\tAEC compression support: {has_aec_support}') 43 print(f'') 44 print(f'NCEPLIPS-ip support: {has_interpolation}') 45 print(f'\tStatic library: {ip_static}') 46 print(f'\tOpenMP support: {has_openmp_support}') 47 print(f'') 48 print(f'Static libs:') 49 for lib in extra_objects: 50 print(f'\t{lib}') 51 print(f'') 52 print(f'NCEP GRIB2 Table Version: {_ncep_grib2_table_version}')
Print grib2io build configuration information.
1698def interpolate(a, 1699 method: Union[int, str], 1700 grid_def_in, 1701 grid_def_out, 1702 method_options=None, 1703 num_threads=1, 1704): 1705 """ 1706 This is the module-level interpolation function. 1707 1708 This interfaces with the 4-byte (32-bit float) [NCEPLIBS-ip library](https://github.com/NOAA-EMC/NCEPLIBS-ip) 1709 through grib2io's internal iplib Cython extension module. 1710 1711 Parameters 1712 ---------- 1713 a : numpy.ndarray or tuple 1714 Input data. If `a` is a `numpy.ndarray`, scalar interpolation will be 1715 performed. If `a` is a `tuple`, then vector interpolation will be 1716 performed with the assumption that u = a[0] and v = a[1] and are both 1717 `numpy.ndarray`. 1718 1719 These data are expected to be in 2-dimensional form with shape (ny, nx) 1720 or 3-dimensional (:, ny, nx) where the 1st dimension represents another 1721 spatial, temporal, or classification (i.e. ensemble members) dimension. 1722 The function will properly flatten the (ny,nx) dimensions into (nx * ny) 1723 acceptable for input into the interpolation subroutines. If needed, these 1724 data will be converted to `np.float32`. 1725 method 1726 Interpolate method to use. This can either be an integer or string using 1727 the following mapping: 1728 1729 | Interpolate Scheme | Integer Value | 1730 | :---: | :---: | 1731 | 'bilinear' | 0 | 1732 | 'bicubic' | 1 | 1733 | 'neighbor' | 2 | 1734 | 'budget' | 3 | 1735 | 'spectral' | 4 | 1736 | 'neighbor-budget' | 6 | 1737 1738 grid_def_in : grib2io.Grib2GridDef 1739 Grib2GridDef object for the input grid. 1740 grid_def_out : grib2io.Grib2GridDef 1741 Grib2GridDef object for the output grid or station points. 1742 method_options : list of ints, optional 1743 Interpolation options. See the NCEPLIBS-ip documentation for 1744 more information on how these are used. 1745 num_threads : int, optional 1746 Number of OpenMP threads to use for interpolation. The default 1747 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1748 was not built with OpenMP, then this keyword argument and value 1749 will have no impact. 1750 1751 Returns 1752 ------- 1753 interpolate 1754 Returns a `numpy.ndarray` of dtype `np.float32` when scalar interpolation 1755 is performed or a `tuple` of `numpy.ndarray`s when vector interpolation is 1756 performed with the assumptions that 0-index is the interpolated u and 1757 1-index is the interpolated v. 1758 """ 1759 1760 from . import iplib 1761 1762 prev_num_threads = 1 1763 try: 1764 prev_num_threads = iplib.openmp_get_num_threads() 1765 iplib.openmp_set_num_threads(num_threads) 1766 except(AttributeError): 1767 pass 1768 1769 print(f"grib2io.interpolate thread report: OpenMP num threads = {iplib.openmp_get_num_threads()}") 1770 1771 if isinstance(method,int) and method not in _interp_schemes.values(): 1772 raise ValueError('Invalid interpolation method.') 1773 elif isinstance(method,str): 1774 if method in _interp_schemes.keys(): 1775 method = _interp_schemes[method] 1776 else: 1777 raise ValueError('Invalid interpolation method.') 1778 1779 if method_options is None: 1780 method_options = np.zeros((20),dtype=np.int32) 1781 if method in {3,6}: 1782 method_options[0:2] = -1 1783 1784 mi = grid_def_in.npoints 1785 mo = grid_def_out.npoints 1786 1787 # Adjust shape of input array(s) 1788 a, newshp = _adjust_array_shape_for_interp(a, grid_def_in, grid_def_out) 1789 1790 # Call interpolation subroutines according to type of a. 1791 if isinstance(a,np.ndarray): 1792 # Scalar 1793 km = a.shape[0] 1794 if np.any(np.isnan(a)): 1795 ibi = np.ones((km), dtype=np.int32) 1796 li = np.where(np.isnan(a),0,1).astype(np.uint8) 1797 else: 1798 ibi = np.zeros((km), dtype=np.int32) 1799 li = np.zeros(a.shape,dtype=np.uint8) 1800 no, rlat, rlon, ibo, lo, go, iret = iplib.interpolate_scalar( 1801 method, 1802 method_options, 1803 grid_def_in.gdtn, 1804 np.array(grid_def_in.gdt, dtype=np.int32), 1805 grid_def_out.gdtn, 1806 np.array(grid_def_out.gdt, dtype=np.int32), 1807 mi, 1808 mo, 1809 km, 1810 ibi, 1811 li, 1812 a, 1813 ) 1814 out = np.where(lo==0,np.nan,go).reshape(newshp) 1815 elif isinstance(a,tuple): 1816 # Vector 1817 km = a[0].shape[0] 1818 if np.any(np.isnan(a)): 1819 ibi = np.ones((km), dtype=np.int32) 1820 li = np.where(np.isnan(a),0,1).astype(np.uint8) 1821 else: 1822 ibi = np.zeros((km), dtype=np.int32) 1823 li = np.zeros(a[0].shape,dtype=np.uint8) 1824 no, rlat, rlon, crot, srot, ibo, lo, uo, vo, iret = iplib.interpolate_vector( 1825 method, 1826 method_options, 1827 grid_def_in.gdtn, 1828 np.array(grid_def_in.gdt, dtype=np.int32), 1829 grid_def_out.gdtn, 1830 np.array(grid_def_out.gdt, dtype=np.int32), 1831 mi, 1832 mo, 1833 km, 1834 ibi, 1835 li, 1836 a[0], 1837 a[1], 1838 ) 1839 uo = np.where(lo==0,np.nan,uo).reshape(newshp) 1840 vo = np.where(lo==0,np.nan,vo).reshape(newshp) 1841 out = (uo,vo) 1842 1843 try: 1844 iplib.openmp_set_num_threads(prev_num_threads) 1845 except(AttributeError): 1846 pass 1847 1848 return out
This is the module-level interpolation function.
This interfaces with the 4-byte (32-bit float) NCEPLIBS-ip library through grib2io's internal iplib Cython extension module.
Parameters
a (numpy.ndarray or tuple): Input data. If
a
is anumpy.ndarray
, scalar interpolation will be performed. Ifa
is atuple
, then vector interpolation will be performed with the assumption that u = a[0] and v = a[1] and are bothnumpy.ndarray
.These data are expected to be in 2-dimensional form with shape (ny, nx) or 3-dimensional (:, ny, nx) where the 1st dimension represents another spatial, temporal, or classification (i.e. ensemble members) dimension. The function will properly flatten the (ny,nx) dimensions into (nx * ny) acceptable for input into the interpolation subroutines. If needed, these data will be converted to
np.float32
.- method: Interpolate method to use. This can either be an integer or string using the following mapping:
Interpolate Scheme | Integer Value |
---|---|
'bilinear' | 0 |
'bicubic' | 1 |
'neighbor' | 2 |
'budget' | 3 |
'spectral' | 4 |
'neighbor-budget' | 6 |
- grid_def_in (grib2io.Grib2GridDef): Grib2GridDef object for the input grid.
- grid_def_out (grib2io.Grib2GridDef): Grib2GridDef object for the output grid or station points.
- method_options (list of ints, optional): Interpolation options. See the NCEPLIBS-ip documentation for more information on how these are used.
- num_threads (int, optional): Number of OpenMP threads to use for interpolation. The default value is 1. If NCEPLIBS-ip and grib2io's iplib extension module was not built with OpenMP, then this keyword argument and value will have no impact.
Returns
- interpolate: Returns a
numpy.ndarray
of dtypenp.float32
when scalar interpolation is performed or atuple
ofnumpy.ndarray
s when vector interpolation is performed with the assumptions that 0-index is the interpolated u and 1-index is the interpolated v.
1851def interpolate_to_stations( 1852 a, 1853 method: Union[int, str], 1854 grid_def_in, 1855 lats, 1856 lons, 1857 method_options=None, 1858 num_threads=1 1859): 1860 """ 1861 Module-level interpolation function for interpolation to stations. 1862 1863 Interfaces with the 4-byte (32-bit float) [NCEPLIBS-ip library](https://github.com/NOAA-EMC/NCEPLIBS-ip) 1864 via grib2io's iplib Cython exntension module. It supports scalar and 1865 vector interpolation according to the type of object a. 1866 1867 Parameters 1868 ---------- 1869 a : numpy.ndarray or tuple 1870 Input data. If `a` is a `numpy.ndarray`, scalar interpolation will be 1871 performed. If `a` is a `tuple`, then vector interpolation will be 1872 performed with the assumption that u = a[0] and v = a[1] and are both 1873 `numpy.ndarray`. 1874 1875 These data are expected to be in 2-dimensional form with shape (ny, nx) 1876 or 3-dimensional (:, ny, nx) where the 1st dimension represents another 1877 spatial, temporal, or classification (i.e. ensemble members) dimension. 1878 The function will properly flatten the (ny,nx) dimensions into (nx * ny) 1879 acceptable for input into the interpolation subroutines. If needed, these 1880 data will be converted to `np.float32`. 1881 method 1882 Interpolate method to use. This can either be an integer or string using 1883 the following mapping: 1884 1885 | Interpolate Scheme | Integer Value | 1886 | :---: | :---: | 1887 | 'bilinear' | 0 | 1888 | 'bicubic' | 1 | 1889 | 'neighbor' | 2 | 1890 | 'budget' | 3 | 1891 | 'spectral' | 4 | 1892 | 'neighbor-budget' | 6 | 1893 1894 grid_def_in : grib2io.Grib2GridDef 1895 Grib2GridDef object for the input grid. 1896 lats : numpy.ndarray or list 1897 Latitudes for station points 1898 lons : numpy.ndarray or list 1899 Longitudes for station points 1900 method_options : list of ints, optional 1901 Interpolation options. See the NCEPLIBS-ip documentation for 1902 more information on how these are used. 1903 num_threads : int, optional 1904 Number of OpenMP threads to use for interpolation. The default 1905 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1906 was not built with OpenMP, then this keyword argument and value 1907 will have no impact. 1908 1909 Returns 1910 ------- 1911 interpolate_to_stations 1912 Returns a `numpy.ndarray` of dtype `np.float32` when scalar 1913 interpolation is performed or a `tuple` of `numpy.ndarray`s 1914 when vector interpolation is performed with the assumptions 1915 that 0-index is the interpolated u and 1-index is the 1916 interpolated v. 1917 """ 1918 from . import iplib 1919 1920 prev_num_threads = 1 1921 try: 1922 prev_num_threads = iplib.openmp_get_num_threads() 1923 iplib.openmp_set_num_threads(num_threads) 1924 except(AttributeError): 1925 pass 1926 1927 if isinstance(method,int) and method not in _interp_schemes.values(): 1928 raise ValueError('Invalid interpolation method.') 1929 elif isinstance(method,str): 1930 if method in _interp_schemes.keys(): 1931 method = _interp_schemes[method] 1932 else: 1933 raise ValueError('Invalid interpolation method.') 1934 1935 if method_options is None: 1936 method_options = np.zeros((20),dtype=np.int32) 1937 if method in {3,6}: 1938 method_options[0:2] = -1 1939 1940 # Check lats and lons 1941 if isinstance(lats,list): 1942 nlats = len(lats) 1943 elif isinstance(lats,np.ndarray) and len(lats.shape) == 1: 1944 nlats = lats.shape[0] 1945 else: 1946 raise ValueError('Station latitudes must be a list or 1-D NumPy array.') 1947 if isinstance(lons,list): 1948 nlons = len(lons) 1949 elif isinstance(lons,np.ndarray) and len(lons.shape) == 1: 1950 nlons = lons.shape[0] 1951 else: 1952 raise ValueError('Station longitudes must be a list or 1-D NumPy array.') 1953 if nlats != nlons: 1954 raise ValueError('Station lats and lons must be same size.') 1955 1956 mi = grid_def_in.npoints 1957 mo = nlats 1958 1959 # Adjust shape of input array(s) 1960 a, newshp = _adjust_array_shape_for_interp_stations(a, grid_def_in, mo) 1961 1962 # Use gdtn = -1 for stations and an empty template array 1963 gdtn = -1 1964 gdt = np.zeros((200),dtype=np.int32) 1965 1966 # Call interpolation subroutines according to type of a. 1967 if isinstance(a,np.ndarray): 1968 # Scalar 1969 km = a.shape[0] 1970 ibi = np.zeros((km), dtype=np.int32) 1971 li = np.zeros(a.shape,dtype=np.uint8) 1972 no, rlat, rlon, ibo, lo, go, iret = iplib.interpolate_scalar( 1973 method, 1974 method_options, 1975 grid_def_in.gdtn, 1976 np.array(grid_def_in.gdt, dtype=np.int32), 1977 gdtn, 1978 gdt, 1979 mi, 1980 mo, 1981 km, 1982 ibi, 1983 li, 1984 a, 1985 lats=np.array(lats, dtype=np.float32), 1986 lons=np.array(lons, dtype=np.float32), 1987 ) 1988 out = go.reshape(newshp) 1989 1990 elif isinstance(a,tuple): 1991 # Vector 1992 km = a[0].shape[0] 1993 ibi = np.zeros((km), dtype=np.int32) 1994 li = np.zeros(a[0].shape,dtype=np.uint8) 1995 no, rlat, rlon, crot, srot, ibo, lo, uo, vo, iret = iplib.interpolate_vector( 1996 method, 1997 method_options, 1998 grid_def_in.gdtn, 1999 np.array(grid_def_in.gdt, dtype=np.int32), 2000 gdtn, 2001 gdt, 2002 mi, 2003 mo, 2004 km, 2005 ibi, 2006 li, 2007 a[0], 2008 a[1], 2009 lats=np.array(lats, dtype=np.float32), 2010 lons=np.array(lons, dtype=np.float32), 2011 ) 2012 out = (uo.reshape(newshp),vo.reshape(newshp)) 2013 2014 try: 2015 iplib.openmp_set_num_threads(prev_num_threads) 2016 except(AttributeError): 2017 pass 2018 2019 return out
Module-level interpolation function for interpolation to stations.
Interfaces with the 4-byte (32-bit float) NCEPLIBS-ip library via grib2io's iplib Cython exntension module. It supports scalar and vector interpolation according to the type of object a.
Parameters
a (numpy.ndarray or tuple): Input data. If
a
is anumpy.ndarray
, scalar interpolation will be performed. Ifa
is atuple
, then vector interpolation will be performed with the assumption that u = a[0] and v = a[1] and are bothnumpy.ndarray
.These data are expected to be in 2-dimensional form with shape (ny, nx) or 3-dimensional (:, ny, nx) where the 1st dimension represents another spatial, temporal, or classification (i.e. ensemble members) dimension. The function will properly flatten the (ny,nx) dimensions into (nx * ny) acceptable for input into the interpolation subroutines. If needed, these data will be converted to
np.float32
.- method: Interpolate method to use. This can either be an integer or string using the following mapping:
Interpolate Scheme | Integer Value |
---|---|
'bilinear' | 0 |
'bicubic' | 1 |
'neighbor' | 2 |
'budget' | 3 |
'spectral' | 4 |
'neighbor-budget' | 6 |
- grid_def_in (grib2io.Grib2GridDef): Grib2GridDef object for the input grid.
- lats (numpy.ndarray or list): Latitudes for station points
- lons (numpy.ndarray or list): Longitudes for station points
- method_options (list of ints, optional): Interpolation options. See the NCEPLIBS-ip documentation for more information on how these are used.
- num_threads (int, optional): Number of OpenMP threads to use for interpolation. The default value is 1. If NCEPLIBS-ip and grib2io's iplib extension module was not built with OpenMP, then this keyword argument and value will have no impact.
Returns
- interpolate_to_stations: Returns a
numpy.ndarray
of dtypenp.float32
when scalar interpolation is performed or atuple
ofnumpy.ndarray
s when vector interpolation is performed with the assumptions that 0-index is the interpolated u and 1-index is the interpolated v.
586class Grib2Message: 587 """ 588 Creation class for a GRIB2 message. 589 590 This class returns a dynamically-created Grib2Message object that 591 inherits from `_Grib2Message` and grid, product, data representation 592 template classes according to the template numbers for the respective 593 sections. If `section3`, `section4`, or `section5` are omitted, then 594 the appropriate keyword arguments for the template number `gdtn=`, 595 `pdtn=`, or `drtn=` must be provided. 596 597 Parameters 598 ---------- 599 section0 600 GRIB2 section 0 array. 601 section1 602 GRIB2 section 1 array. 603 section2 604 Local Use section data. 605 section3 606 GRIB2 section 3 array. 607 section4 608 GRIB2 section 4 array. 609 section5 610 GRIB2 section 5 array. 611 612 Returns 613 ------- 614 Msg 615 A dynamically-create Grib2Message object that inherits from 616 _Grib2Message, a grid definition template class, product 617 definition template class, and a data representation template 618 class. 619 """ 620 def __new__(self, section0: NDArray = np.array([struct.unpack('>I',b'GRIB')[0],0,0,2,0]), 621 section1: NDArray = np.zeros((13),dtype=np.int64), 622 section2: Optional[bytes] = None, 623 section3: Optional[NDArray] = None, 624 section4: Optional[NDArray] = None, 625 section5: Optional[NDArray] = None, *args, **kwargs): 626 627 if np.all(section1==0): 628 try: 629 # Python >= 3.10 630 section1[5:11] = datetime.datetime.fromtimestamp(0, datetime.UTC).timetuple()[:6] 631 except(AttributeError): 632 # Python < 3.10 633 section1[5:11] = datetime.datetime.utcfromtimestamp(0).timetuple()[:6] 634 635 bases = list() 636 if section3 is None: 637 if 'gdtn' in kwargs.keys(): 638 gdtn = kwargs['gdtn'] 639 Gdt = templates.gdt_class_by_gdtn(gdtn) 640 bases.append(Gdt) 641 section3 = np.zeros((Gdt._len+5),dtype=np.int64) 642 section3[4] = gdtn 643 else: 644 raise ValueError("Must provide GRIB2 Grid Definition Template Number or section 3 array") 645 else: 646 gdtn = section3[4] 647 Gdt = templates.gdt_class_by_gdtn(gdtn) 648 bases.append(Gdt) 649 650 if section4 is None: 651 if 'pdtn' in kwargs.keys(): 652 pdtn = kwargs['pdtn'] 653 Pdt = templates.pdt_class_by_pdtn(pdtn) 654 bases.append(Pdt) 655 section4 = np.zeros((Pdt._len+2),dtype=np.int64) 656 section4[1] = pdtn 657 else: 658 raise ValueError("Must provide GRIB2 Production Definition Template Number or section 4 array") 659 else: 660 pdtn = section4[1] 661 Pdt = templates.pdt_class_by_pdtn(pdtn) 662 bases.append(Pdt) 663 664 if section5 is None: 665 if 'drtn' in kwargs.keys(): 666 drtn = kwargs['drtn'] 667 Drt = templates.drt_class_by_drtn(drtn) 668 bases.append(Drt) 669 section5 = np.zeros((Drt._len+2),dtype=np.int64) 670 section5[1] = drtn 671 else: 672 raise ValueError("Must provide GRIB2 Data Representation Template Number or section 5 array") 673 else: 674 drtn = section5[1] 675 Drt = templates.drt_class_by_drtn(drtn) 676 bases.append(Drt) 677 678 # attempt to use existing Msg class if it has already been made with gdtn,pdtn,drtn combo 679 try: 680 Msg = _msg_class_store[f"{gdtn}:{pdtn}:{drtn}"] 681 except KeyError: 682 @dataclass(init=False, repr=False) 683 class Msg(_Grib2Message, *bases): 684 pass 685 _msg_class_store[f"{gdtn}:{pdtn}:{drtn}"] = Msg 686 687 return Msg(section0, section1, section2, section3, section4, section5, *args)
Creation class for a GRIB2 message.
This class returns a dynamically-created Grib2Message object that
inherits from _Grib2Message
and grid, product, data representation
template classes according to the template numbers for the respective
sections. If section3
, section4
, or section5
are omitted, then
the appropriate keyword arguments for the template number gdtn=
,
pdtn=
, or drtn=
must be provided.
Parameters
- section0: GRIB2 section 0 array.
- section1: GRIB2 section 1 array.
- section2: Local Use section data.
- section3: GRIB2 section 3 array.
- section4: GRIB2 section 4 array.
- section5: GRIB2 section 5 array.
Returns
- Msg: A dynamically-create Grib2Message object that inherits from _Grib2Message, a grid definition template class, product definition template class, and a data representation template class.
690@dataclass 691class _Grib2Message: 692 """ 693 GRIB2 Message base class. 694 """ 695 # GRIB2 Sections 696 section0: NDArray = field(init=True,repr=False) 697 section1: NDArray = field(init=True,repr=False) 698 section2: bytes = field(init=True,repr=False) 699 section3: NDArray = field(init=True,repr=False) 700 section4: NDArray = field(init=True,repr=False) 701 section5: NDArray = field(init=True,repr=False) 702 bitMapFlag: templates.Grib2Metadata = field(init=True,repr=False,default=255) 703 704 # Section 0 looked up attributes 705 indicatorSection: NDArray = field(init=False,repr=False,default=templates.IndicatorSection()) 706 discipline: templates.Grib2Metadata = field(init=False,repr=False,default=templates.Discipline()) 707 708 # Section 1 looked up attributes 709 identificationSection: NDArray = field(init=False,repr=False,default=templates.IdentificationSection()) 710 originatingCenter: templates.Grib2Metadata = field(init=False,repr=False,default=templates.OriginatingCenter()) 711 originatingSubCenter: templates.Grib2Metadata = field(init=False,repr=False,default=templates.OriginatingSubCenter()) 712 masterTableInfo: templates.Grib2Metadata = field(init=False,repr=False,default=templates.MasterTableInfo()) 713 localTableInfo: templates.Grib2Metadata = field(init=False,repr=False,default=templates.LocalTableInfo()) 714 significanceOfReferenceTime: templates.Grib2Metadata = field(init=False,repr=False,default=templates.SignificanceOfReferenceTime()) 715 year: int = field(init=False,repr=False,default=templates.Year()) 716 month: int = field(init=False,repr=False,default=templates.Month()) 717 day: int = field(init=False,repr=False,default=templates.Day()) 718 hour: int = field(init=False,repr=False,default=templates.Hour()) 719 minute: int = field(init=False,repr=False,default=templates.Minute()) 720 second: int = field(init=False,repr=False,default=templates.Second()) 721 refDate: datetime.datetime = field(init=False,repr=False,default=templates.RefDate()) 722 productionStatus: templates.Grib2Metadata = field(init=False,repr=False,default=templates.ProductionStatus()) 723 typeOfData: templates.Grib2Metadata = field(init=False,repr=False,default=templates.TypeOfData()) 724 725 # Section 3 looked up common attributes. Other looked up attributes are available according 726 # to the Grid Definition Template. 727 gridDefinitionSection: NDArray = field(init=False,repr=False,default=templates.GridDefinitionSection()) 728 sourceOfGridDefinition: int = field(init=False,repr=False,default=templates.SourceOfGridDefinition()) 729 numberOfDataPoints: int = field(init=False,repr=False,default=templates.NumberOfDataPoints()) 730 interpretationOfListOfNumbers: templates.Grib2Metadata = field(init=False,repr=False,default=templates.InterpretationOfListOfNumbers()) 731 gridDefinitionTemplateNumber: templates.Grib2Metadata = field(init=False,repr=False,default=templates.GridDefinitionTemplateNumber()) 732 gridDefinitionTemplate: list = field(init=False,repr=False,default=templates.GridDefinitionTemplate()) 733 _earthparams: dict = field(init=False,repr=False,default=templates.EarthParams()) 734 _dxsign: float = field(init=False,repr=False,default=templates.DxSign()) 735 _dysign: float = field(init=False,repr=False,default=templates.DySign()) 736 _llscalefactor: float = field(init=False,repr=False,default=templates.LLScaleFactor()) 737 _lldivisor: float = field(init=False,repr=False,default=templates.LLDivisor()) 738 _xydivisor: float = field(init=False,repr=False,default=templates.XYDivisor()) 739 shapeOfEarth: templates.Grib2Metadata = field(init=False,repr=False,default=templates.ShapeOfEarth()) 740 earthShape: str = field(init=False,repr=False,default=templates.EarthShape()) 741 earthRadius: float = field(init=False,repr=False,default=templates.EarthRadius()) 742 earthMajorAxis: float = field(init=False,repr=False,default=templates.EarthMajorAxis()) 743 earthMinorAxis: float = field(init=False,repr=False,default=templates.EarthMinorAxis()) 744 resolutionAndComponentFlags: list = field(init=False,repr=False,default=templates.ResolutionAndComponentFlags()) 745 ny: int = field(init=False,repr=False,default=templates.Ny()) 746 nx: int = field(init=False,repr=False,default=templates.Nx()) 747 scanModeFlags: list = field(init=False,repr=False,default=templates.ScanModeFlags()) 748 projParameters: dict = field(init=False,repr=False,default=templates.ProjParameters()) 749 750 # Section 4 751 productDefinitionTemplateNumber: templates.Grib2Metadata = field(init=False,repr=False,default=templates.ProductDefinitionTemplateNumber()) 752 productDefinitionTemplate: NDArray = field(init=False,repr=False,default=templates.ProductDefinitionTemplate()) 753 754 # Section 5 looked up common attributes. Other looked up attributes are 755 # available according to the Data Representation Template. 756 numberOfPackedValues: int = field(init=False,repr=False,default=templates.NumberOfPackedValues()) 757 dataRepresentationTemplateNumber: templates.Grib2Metadata = field(init=False,repr=False,default=templates.DataRepresentationTemplateNumber()) 758 dataRepresentationTemplate: list = field(init=False,repr=False,default=templates.DataRepresentationTemplate()) 759 typeOfValues: templates.Grib2Metadata = field(init=False,repr=False,default=templates.TypeOfValues()) 760 761 def __post_init__(self): 762 """Set some attributes after init.""" 763 self._auto_nans = _AUTO_NANS 764 self._coordlist = np.zeros((0), dtype=np.float32) 765 self._data = None 766 self._deflist = np.zeros((0), dtype=np.int64) 767 self._msgnum = -1 768 self._ondiskarray = None 769 self._orig_section5 = np.copy(self.section5) 770 self._signature = self._generate_signature() 771 try: 772 self._sha1_section3 = hashlib.sha1(self.section3).hexdigest() 773 except(TypeError): 774 pass 775 self.bitMapFlag = templates.Grib2Metadata(self.bitMapFlag,table='6.0') 776 self.bitmap = None 777 778 @property 779 def _isNDFD(self): 780 """Check if GRIB2 message is from NWS NDFD""" 781 return np.all(self.section1[0:2]==[8,65535]) 782 783 @property 784 def _isAerosol(self): 785 """Check if GRIB2 message contains aerosol data""" 786 is_aero_template = self.productDefinitionTemplateNumber.value in tables.AEROSOL_PDTNS 787 is_aero_param = ((str(self.parameterCategory) == '13') | 788 (str(self.parameterCategory) == '20')) and str(self.parameterNumber) in tables.AEROSOL_PARAMS 789 # Check table 4.205 aerosol presence 790 is_aero_type = (str(self.parameterCategory) == '205' and 791 str(self.parameterNumber) == '1') 792 return is_aero_template or is_aero_param or is_aero_type 793 794 @property 795 def gdtn(self): 796 """Return Grid Definition Template Number""" 797 return self.section3[4] 798 799 800 @property 801 def gdt(self): 802 """Return Grid Definition Template.""" 803 return self.gridDefinitionTemplate 804 805 806 @property 807 def pdtn(self): 808 """Return Product Definition Template Number.""" 809 return self.section4[1] 810 811 812 @property 813 def pdt(self): 814 """Return Product Definition Template.""" 815 return self.productDefinitionTemplate 816 817 818 @property 819 def drtn(self): 820 """Return Data Representation Template Number.""" 821 return self.section5[1] 822 823 824 @property 825 def drt(self): 826 """Return Data Representation Template.""" 827 return self.dataRepresentationTemplate 828 829 830 @property 831 def pdy(self): 832 """Return the PDY ('YYYYMMDD').""" 833 return ''.join([str(i) for i in self.section1[5:8]]) 834 835 836 @property 837 def griddef(self): 838 """Return a Grib2GridDef instance for a GRIB2 message.""" 839 return Grib2GridDef.from_section3(self.section3) 840 841 842 @property 843 def lats(self): 844 """Return grid latitudes.""" 845 return self.latlons()[0] 846 847 848 @property 849 def lons(self): 850 """Return grid longitudes.""" 851 return self.latlons()[1] 852 853 @property 854 def min(self): 855 """Return minimum value of data.""" 856 return np.nanmin(self.data) 857 858 859 @property 860 def max(self): 861 """Return maximum value of data.""" 862 return np.nanmax(self.data) 863 864 865 @property 866 def mean(self): 867 """Return mean value of data.""" 868 return np.nanmean(self.data) 869 870 871 @property 872 def median(self): 873 """Return median value of data.""" 874 return np.nanmedian(self.data) 875 876 877 @property 878 def shape(self): 879 """Return shape of data.""" 880 return self.griddef.shape 881 882 883 def __repr__(self): 884 """ 885 Return an unambiguous string representation of the object. 886 887 Returns 888 ------- 889 repr 890 A string representation of the object, including information from 891 sections 0, 1, 3, 4, 5, and 6. 892 """ 893 info = '' 894 for sect in [0,1,3,4,5,6]: 895 for k,v in self.attrs_by_section(sect,values=True).items(): 896 info += f'Section {sect}: {k} = {v}\n' 897 return info 898 899 def __str__(self): 900 """ 901 Return a readable string representation of the object. 902 903 Returns 904 ------- 905 str 906 A formatted string representation of the object, including 907 selected attributes. 908 """ 909 return (f'{self._msgnum}:d={self.refDate}:{self.shortName}:' 910 f'{self.fullName} ({self.units}):{self.level}:' 911 f'{self.leadTime}') 912 913 914 def _generate_signature(self): 915 """Generature SHA-1 hash string from GRIB2 integer sections.""" 916 return hashlib.sha1(np.concatenate((self.section0,self.section1, 917 self.section3,self.section4, 918 self.section5))).hexdigest() 919 920 921 def attrs_by_section(self, sect: int, values: bool=False): 922 """ 923 Provide a tuple of attribute names for the given GRIB2 section. 924 925 Parameters 926 ---------- 927 sect 928 The GRIB2 section number. 929 values 930 Optional (default is `False`) argument to return attributes values. 931 932 Returns 933 ------- 934 attrs_by_section 935 A list of attribute names or dict of name:value pairs if `values = 936 True`. 937 """ 938 if sect in {0,1,6}: 939 attrs = templates._section_attrs[sect] 940 elif sect in {3,4,5}: 941 def _find_class_index(n): 942 _key = {3:'Grid', 4:'Product', 5:'Data'} 943 for i,c in enumerate(self.__class__.__mro__): 944 if _key[n] in c.__name__: 945 return i 946 else: 947 return [] 948 if sys.version_info.minor <= 8: 949 attrs = templates._section_attrs[sect]+\ 950 [a for a in dir(self.__class__.__mro__[_find_class_index(sect)]) if not a.startswith('_')] 951 else: 952 attrs = templates._section_attrs[sect]+\ 953 self.__class__.__mro__[_find_class_index(sect)]._attrs() 954 else: 955 attrs = [] 956 if values: 957 return {k:getattr(self,k) for k in attrs} 958 else: 959 return attrs 960 961 962 def pack(self): 963 """ 964 Pack GRIB2 section data into a binary message. 965 966 It is the user's responsibility to populate the GRIB2 section 967 information with appropriate metadata. 968 """ 969 # Create beginning of packed binary message with section 0 and 1 data. 970 self._sections = [] 971 self._msg,self._pos = g2clib.grib2_create(self.indicatorSection[2:4],self.identificationSection) 972 self._sections += [0,1] 973 974 # Add section 2 if present. 975 if isinstance(self.section2,bytes) and len(self.section2) > 0: 976 self._msg,self._pos = g2clib.grib2_addlocal(self._msg,self.section2) 977 self._sections.append(2) 978 979 # Add section 3. 980 self.section3[1] = self.nx * self.ny 981 self._msg,self._pos = g2clib.grib2_addgrid(self._msg,self.gridDefinitionSection, 982 self.gridDefinitionTemplate, 983 self._deflist) 984 self._sections.append(3) 985 986 # Prepare data. 987 if self._data is None: 988 if self._ondiskarray is None: 989 raise ValueError("Grib2Message object has no data, thus it cannot be packed.") 990 field = np.copy(self.data) 991 if self.scanModeFlags is not None: 992 if self.scanModeFlags[3]: 993 fieldsave = field.astype('f') # Casting makes a copy 994 field[1::2,:] = fieldsave[1::2,::-1] 995 fld = field.astype('f') 996 997 # Prepare bitmap, if necessary 998 bitmapflag = self.bitMapFlag.value 999 if bitmapflag == 0: 1000 if self.bitmap is not None: 1001 bmap = np.ravel(self.bitmap).astype(DEFAULT_NUMPY_INT) 1002 else: 1003 bmap = np.ravel(np.where(np.isnan(fld),0,1)).astype(DEFAULT_NUMPY_INT) 1004 else: 1005 bmap = None 1006 1007 # Prepare data for packing if nans are present 1008 fld = np.ravel(fld) 1009 if bitmapflag in {0,254}: 1010 fld = np.where(np.isnan(fld),0,fld) 1011 else: 1012 if np.isnan(fld).any(): 1013 if hasattr(self,'priMissingValue'): 1014 fld = np.where(np.isnan(fld),self.priMissingValue,fld) 1015 if hasattr(self,'_missvalmap'): 1016 if hasattr(self,'priMissingValue'): 1017 fld = np.where(self._missvalmap==1,self.priMissingValue,fld) 1018 if hasattr(self,'secMissingValue'): 1019 fld = np.where(self._missvalmap==2,self.secMissingValue,fld) 1020 1021 # Add sections 4, 5, 6, and 7. 1022 self._msg,self._pos = g2clib.grib2_addfield(self._msg,self.pdtn, 1023 self.productDefinitionTemplate, 1024 self._coordlist, 1025 self.drtn, 1026 self.dataRepresentationTemplate, 1027 fld, 1028 bitmapflag, 1029 bmap) 1030 self._sections.append(4) 1031 self._sections.append(5) 1032 self._sections.append(6) 1033 self._sections.append(7) 1034 1035 # Finalize GRIB2 message with section 8. 1036 self._msg, self._pos = g2clib.grib2_end(self._msg) 1037 self._sections.append(8) 1038 self.section0[-1] = len(self._msg) 1039 1040 1041 @property 1042 def data(self) -> np.array: 1043 """Access the unpacked data values.""" 1044 if self._data is None: 1045 if self._auto_nans != _AUTO_NANS: 1046 self._data = self._ondiskarray 1047 self._data = np.asarray(self._ondiskarray) 1048 return self._data 1049 1050 1051 @data.setter 1052 def data(self, data): 1053 if not isinstance(data, np.ndarray): 1054 raise ValueError('Grib2Message data only supports numpy arrays') 1055 self._data = data 1056 1057 1058 def flush_data(self): 1059 """ 1060 Flush the unpacked data values from the Grib2Message object. 1061 1062 Note: If the Grib2Message object was constructed from "scratch" (i.e. 1063 not read from file), this method will remove the data array from 1064 the object and it cannot be recovered. 1065 """ 1066 self._data = None 1067 self.bitmap = None 1068 1069 1070 def __getitem__(self, item): 1071 return self.data[item] 1072 1073 1074 def __setitem__(self, item): 1075 raise NotImplementedError('assignment of data not supported via setitem') 1076 1077 1078 def latlons(self, *args, **kwrgs): 1079 """Alias for `grib2io.Grib2Message.grid` method.""" 1080 return self.grid(*args, **kwrgs) 1081 1082 1083 def grid(self, unrotate: bool=True): 1084 """ 1085 Return lats,lons (in degrees) of grid. 1086 1087 Currently can handle reg. lat/lon,cglobal Gaussian, mercator, 1088 stereographic, lambert conformal, albers equal-area, space-view and 1089 azimuthal equidistant grids. 1090 1091 Parameters 1092 ---------- 1093 unrotate 1094 If `True` [DEFAULT], and grid is rotated lat/lon, then unrotate the 1095 grid, otherwise `False`, do not. 1096 1097 Returns 1098 ------- 1099 lats, lons : numpy.ndarray 1100 Returns two numpy.ndarrays with dtype=numpy.float32 of grid 1101 latitudes and longitudes in units of degrees. 1102 """ 1103 if self._sha1_section3 in _latlon_datastore.keys(): 1104 return (_latlon_datastore[self._sha1_section3]['latitude'], 1105 _latlon_datastore[self._sha1_section3]['longitude']) 1106 gdtn = self.gridDefinitionTemplateNumber.value 1107 gdtmpl = self.gridDefinitionTemplate 1108 reggrid = self.gridDefinitionSection[2] == 0 # This means regular 2-d grid 1109 if gdtn == 0: 1110 # Regular lat/lon grid 1111 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1112 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1113 dlon = self.gridlengthXDirection 1114 dlat = self.gridlengthYDirection 1115 if lon2 < lon1 and dlon < 0: lon1 = -lon1 1116 lats = np.linspace(lat1,lat2,self.ny) 1117 if reggrid: 1118 lons = np.linspace(lon1,lon2,self.nx) 1119 else: 1120 lons = np.linspace(lon1,lon2,self.ny*2) 1121 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1122 elif gdtn == 1: # Rotated Lat/Lon grid 1123 pj = pyproj.Proj(self.projParameters) 1124 lat1,lon1 = self.latitudeFirstGridpoint,self.longitudeFirstGridpoint 1125 lat2,lon2 = self.latitudeLastGridpoint,self.longitudeLastGridpoint 1126 if lon1 > 180.0: lon1 -= 360.0 1127 if lon2 > 180.0: lon2 -= 360.0 1128 lats = np.linspace(lat1,lat2,self.ny) 1129 lons = np.linspace(lon1,lon2,self.nx) 1130 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1131 if unrotate: 1132 from grib2io.utils import rotated_grid 1133 lats,lons = rotated_grid.unrotate(lats,lons,self.anglePoleRotation, 1134 self.latitudeSouthernPole, 1135 self.longitudeSouthernPole) 1136 elif gdtn == 40: # Gaussian grid (only works for global!) 1137 from grib2io.utils.gauss_grid import gaussian_latitudes 1138 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1139 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1140 nlats = self.ny 1141 if not reggrid: # Reduced Gaussian grid. 1142 nlons = 2*nlats 1143 dlon = 360./nlons 1144 else: 1145 nlons = self.nx 1146 dlon = self.gridlengthXDirection 1147 lons = np.linspace(lon1,lon2,nlons) 1148 # Compute Gaussian lats (north to south) 1149 lats = gaussian_latitudes(nlats) 1150 if lat1 < lat2: # reverse them if necessary 1151 lats = lats[::-1] 1152 lons,lats = np.meshgrid(lons,lats) 1153 elif gdtn in {10,20,30,31,110}: 1154 # Mercator, Lambert Conformal, Stereographic, Albers Equal Area, 1155 # Azimuthal Equidistant 1156 dx,dy = self.gridlengthXDirection, self.gridlengthYDirection 1157 lon1,lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1158 pj = pyproj.Proj(self.projParameters) 1159 llcrnrx, llcrnry = pj(lon1,lat1) 1160 x = llcrnrx+dx*np.arange(self.nx) 1161 y = llcrnry+dy*np.arange(self.ny) 1162 x,y = np.meshgrid(x, y) 1163 lons,lats = pj(x, y, inverse=True) 1164 elif gdtn == 90: 1165 # Satellite Projection 1166 dx = self.gridlengthXDirection 1167 dy = self.gridlengthYDirection 1168 pj = pyproj.Proj(self.projParameters) 1169 x = dx*np.indices((self.ny,self.nx),'f')[1,:,:] 1170 x -= 0.5*x.max() 1171 y = dy*np.indices((self.ny,self.nx),'f')[0,:,:] 1172 y -= 0.5*y.max() 1173 lons,lats = pj(x,y,inverse=True) 1174 # Set lons,lats to 1.e30 where undefined 1175 abslons = np.fabs(lons) 1176 abslats = np.fabs(lats) 1177 lons = np.where(abslons < 1.e20, lons, 1.e30) 1178 lats = np.where(abslats < 1.e20, lats, 1.e30) 1179 elif gdtn == 32769: 1180 # Special NCEP Grid, Rotated Lat/Lon, Arakawa E-Grid (Non-Staggered) 1181 from grib2io.utils import arakawa_rotated_grid 1182 from grib2io.utils.rotated_grid import DEG2RAD 1183 di, dj = 0.0, 0.0 1184 do_180 = False 1185 idir = 1 if self.scanModeFlags[0] == 0 else -1 1186 jdir = -1 if self.scanModeFlags[1] == 0 else 1 1187 grid_oriented = 0 if self.resolutionAndComponentFlags[4] == 0 else 1 1188 do_rot = 1 if grid_oriented == 1 else 0 1189 la1 = self.latitudeFirstGridpoint 1190 lo1 = self.longitudeFirstGridpoint 1191 clon = self.longitudeCenterGridpoint 1192 clat = self.latitudeCenterGridpoint 1193 lasp = clat - 90.0 1194 losp = clon 1195 llat, llon = arakawa_rotated_grid.ll2rot(la1,lo1,lasp,losp) 1196 la2, lo2 = arakawa_rotated_grid.rot2ll(-llat,-llon,lasp,losp) 1197 rlat = -llat 1198 rlon = -llon 1199 if self.nx == 1: 1200 di = 0.0 1201 elif idir == 1: 1202 ti = rlon 1203 while ti < llon: 1204 ti += 360.0 1205 di = (ti - llon)/float(self.nx-1) 1206 else: 1207 ti = llon 1208 while ti < rlon: 1209 ti += 360.0 1210 di = (ti - rlon)/float(self.nx-1) 1211 if self.ny == 1: 1212 dj = 0.0 1213 else: 1214 dj = (rlat - llat)/float(self.ny-1) 1215 if dj < 0.0: 1216 dj = -dj 1217 if idir == 1: 1218 if llon > rlon: 1219 llon -= 360.0 1220 if llon < 0 and rlon > 0: 1221 do_180 = True 1222 else: 1223 if rlon > llon: 1224 rlon -= 360.0 1225 if rlon < 0 and llon > 0: 1226 do_180 = True 1227 xlat1d = llat + (np.arange(self.ny)*jdir*dj) 1228 xlon1d = llon + (np.arange(self.nx)*idir*di) 1229 xlons, xlats = np.meshgrid(xlon1d,xlat1d) 1230 rot2ll_vectorized = np.vectorize(arakawa_rotated_grid.rot2ll) 1231 lats, lons = rot2ll_vectorized(xlats,xlons,lasp,losp) 1232 if do_180: 1233 lons = np.where(lons>180.0,lons-360.0,lons) 1234 vector_rotation_angles_vectorized = np.vectorize(arakawa_rotated_grid.vector_rotation_angles) 1235 rots = vector_rotation_angles_vectorized(lats, lons, clat, losp, xlats) 1236 del xlat1d, xlon1d, xlats, xlons 1237 else: 1238 raise ValueError('Unsupported grid') 1239 1240 _latlon_datastore[self._sha1_section3] = dict(latitude=lats,longitude=lons) 1241 try: 1242 _latlon_datastore[self._sha1_section3]['vector_rotation_angles'] = rots 1243 except(NameError): 1244 pass 1245 1246 return lats, lons 1247 1248 1249 def map_keys(self): 1250 """ 1251 Unpack data grid replacing integer values with strings. 1252 1253 These types of fields are categorical or classifications where data 1254 values do not represent an observable or predictable physical quantity. 1255 An example of such a field would be [Dominant Precipitation Type - 1256 DPTYPE](https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table4-201.shtml) 1257 1258 Returns 1259 ------- 1260 map_keys 1261 numpy.ndarray of string values per element. 1262 """ 1263 hold_auto_nans = _AUTO_NANS 1264 set_auto_nans(False) 1265 1266 if (np.all(self.section1[0:2] == [7, 14]) and self.shortName == "PWTHER") or ( 1267 self._isNDFD and self.shortName in {"WX", "WWA"} 1268 ): 1269 keys = utils.decode_wx_strings(self.section2) 1270 if hasattr(self,'priMissingValue') and self.priMissingValue not in [None,0]: 1271 keys[int(self.priMissingValue)] = 'Missing' 1272 if hasattr(self,'secMissingValue') and self.secMissingValue not in [None,0]: 1273 keys[int(self.secMissingValue)] = 'Missing' 1274 u,inv = np.unique(self.data,return_inverse=True) 1275 fld = np.array([keys[x] for x in u])[inv].reshape(self.data.shape) 1276 else: 1277 # For data whose units are defined in a code table (i.e. classification or mask) 1278 tblname = re.findall(r'\d\.\d+',self.units,re.IGNORECASE)[0] 1279 fld = self.data.astype(np.int32).astype(str) 1280 tbl = tables.get_table(tblname,expand=True) 1281 for val in np.unique(fld): 1282 fld = np.where(fld==val,tbl[val],fld) 1283 set_auto_nans(hold_auto_nans) 1284 return fld 1285 1286 1287 def to_bytes(self, validate: bool=True): 1288 """ 1289 Return packed GRIB2 message in bytes format. 1290 1291 This will be useful for exporting data in non-file formats. For example, 1292 can be used to output grib data directly to S3 using the boto3 client 1293 without the need to write a temporary file to upload first. 1294 1295 Parameters 1296 ---------- 1297 validate: default=True 1298 If `True` (DEFAULT), validates first/last four bytes for proper 1299 formatting, else returns None. If `False`, message is output as is. 1300 1301 Returns 1302 ------- 1303 to_bytes 1304 Returns GRIB2 formatted message as bytes. 1305 """ 1306 if hasattr(self,'_msg'): 1307 if validate: 1308 if self.validate(): 1309 return self._msg 1310 else: 1311 return None 1312 else: 1313 return self._msg 1314 else: 1315 return None 1316 1317 1318 def interpolate(self, method, grid_def_out, method_options=None, drtn=None, 1319 num_threads=1): 1320 """ 1321 Grib2Message Interpolator 1322 1323 Performs spatial interpolation via the [NCEPLIBS-ip 1324 library](https://github.com/NOAA-EMC/NCEPLIBS-ip). This interpolate 1325 method only supports scalar interpolation. If you need to perform 1326 vector interpolation, use the module-level `grib2io.interpolate` 1327 function. 1328 1329 Parameters 1330 ---------- 1331 method 1332 Interpolate method to use. This can either be an integer or string 1333 using the following mapping: 1334 1335 | Interpolate Scheme | Integer Value | 1336 | :---: | :---: | 1337 | 'bilinear' | 0 | 1338 | 'bicubic' | 1 | 1339 | 'neighbor' | 2 | 1340 | 'budget' | 3 | 1341 | 'spectral' | 4 | 1342 | 'neighbor-budget' | 6 | 1343 1344 grid_def_out : grib2io.Grib2GridDef 1345 Grib2GridDef object of the output grid. 1346 method_options : list of ints, optional 1347 Interpolation options. See the NCEPLIBS-ip documentation for 1348 more information on how these are used. 1349 drtn 1350 Data Representation Template to be used for the returned 1351 interpolated GRIB2 message. When `None`, the data representation 1352 template of the source GRIB2 message is used. Once again, it is the 1353 user's responsibility to properly set the Data Representation 1354 Template attributes. 1355 num_threads : int, optional 1356 Number of OpenMP threads to use for interpolation. The default 1357 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1358 was not built with OpenMP, then this keyword argument and value 1359 will have no impact. 1360 1361 Returns 1362 ------- 1363 interpolate 1364 If interpolating to a grid, a new Grib2Message object is returned. 1365 The GRIB2 metadata of the new Grib2Message object is identical to 1366 the input except where required to be different because of the new 1367 grid specs and possibly a new data representation template. 1368 1369 If interpolating to station points, the interpolated data values are 1370 returned as a numpy.ndarray. 1371 """ 1372 section0 = self.section0 1373 section0[-1] = 0 1374 gds = [0, grid_def_out.npoints, 0, 255, grid_def_out.gdtn] 1375 section3 = np.concatenate((gds,grid_def_out.gdt)) 1376 drtn = self.drtn if drtn is None else drtn 1377 1378 msg = Grib2Message(section0,self.section1,self.section2,section3, 1379 self.section4,None,self.bitMapFlag.value,drtn=drtn) 1380 1381 msg._msgnum = -1 1382 msg._deflist = self._deflist 1383 msg._coordlist = self._coordlist 1384 shape = (msg.ny,msg.nx) 1385 ndim = 2 1386 if msg.typeOfValues == 0: 1387 dtype = 'float32' 1388 elif msg.typeOfValues == 1: 1389 dtype = 'int32' 1390 msg._data = interpolate(self.data,method,Grib2GridDef.from_section3(self.section3),grid_def_out, 1391 method_options=method_options,num_threads=num_threads).reshape(msg.ny,msg.nx) 1392 msg.section5[0] = grid_def_out.npoints 1393 return msg 1394 1395 1396 def subset(self, lats, lons): 1397 """ 1398 Return a spatial subset. 1399 1400 Currently only supports regular grids of the following types: 1401 1402 | Grid Type | gdtn | 1403 | :---: | :---: | 1404 | Latitude/Longitude, Equidistant Cylindrical, or Plate Carree | 0 | 1405 | Rotated Latitude/Longitude | 1 | 1406 | Mercator | 10 | 1407 | Polar Stereographic | 20 | 1408 | Lambert Conformal | 30 | 1409 | Albers Equal-Area | 31 | 1410 | Gaussian Latitude/Longitude | 40 | 1411 | Equatorial Azimuthal Equidistant Projection | 110 | 1412 1413 Parameters 1414 ---------- 1415 lats 1416 List or tuple of latitudes. The minimum and maximum latitudes will 1417 be used to define the southern and northern boundaries. 1418 1419 The order of the latitudes is not important. The function will 1420 determine which is the minimum and maximum. 1421 1422 The latitudes should be in decimal degrees with 0.0 at the equator, 1423 positive values in the northern hemisphere increasing to 90, and 1424 negative values in the southern hemisphere decreasing to -90. 1425 lons 1426 List or tuple of longitudes. The minimum and maximum longitudes 1427 will be used to define the western and eastern boundaries. 1428 1429 The order of the longitudes is not important. The function will 1430 determine which is the minimum and maximum. 1431 1432 GRIB2 longitudes should be in decimal degrees with 0.0 at the prime 1433 meridian, positive values increasing eastward to 360. There are no 1434 negative GRIB2 longitudes. 1435 1436 The typical west longitudes that start at 0.0 at the prime meridian 1437 and decrease to -180 westward, are converted to GRIB2 longitudes by 1438 '360 - (absolute value of the west longitude)' where typical 1439 eastern longitudes are unchanged as GRIB2 longitudes. 1440 1441 Returns 1442 ------- 1443 subset 1444 A spatial subset of a GRIB2 message. 1445 """ 1446 if self.gdtn not in [0, 1, 10, 20, 30, 31, 40, 110]: 1447 raise ValueError( 1448 """ 1449 1450Subset only works for 1451 Latitude/Longitude, Equidistant Cylindrical, or Plate Carree (gdtn=0) 1452 Rotated Latitude/Longitude (gdtn=1) 1453 Mercator (gdtn=10) 1454 Polar Stereographic (gdtn=20) 1455 Lambert Conformal (gdtn=30) 1456 Albers Equal-Area (gdtn=31) 1457 Gaussian Latitude/Longitude (gdtn=40) 1458 Equatorial Azimuthal Equidistant Projection (gdtn=110) 1459 1460""" 1461 ) 1462 1463 if self.nx == 0 or self.ny == 0: 1464 raise ValueError( 1465 """ 1466 1467Subset only works for regular grids. 1468 1469""" 1470 ) 1471 1472 newmsg = Grib2Message( 1473 np.copy(self.section0), 1474 np.copy(self.section1), 1475 np.copy(self.section2), 1476 np.copy(self.section3), 1477 np.copy(self.section4), 1478 np.copy(self.section5), 1479 ) 1480 1481 msglats, msglons = self.grid() 1482 1483 la1 = np.max(lats) 1484 lo1 = np.min(lons) 1485 la2 = np.min(lats) 1486 lo2 = np.max(lons) 1487 1488 # Find the indices of the first and last grid points to the nearest 1489 # lat/lon values in the grid. 1490 first_lat = np.abs(msglats - la1) 1491 first_lon = np.abs(msglons - lo1) 1492 max_idx = np.maximum(first_lat, first_lon) 1493 first_j, first_i = np.where(max_idx == np.min(max_idx)) 1494 1495 last_lat = np.abs(msglats - la2) 1496 last_lon = np.abs(msglons - lo2) 1497 max_idx = np.maximum(last_lat, last_lon) 1498 last_j, last_i = np.where(max_idx == np.min(max_idx)) 1499 1500 setattr(newmsg, "latitudeFirstGridpoint", msglats[first_j[0], first_i[0]]) 1501 setattr(newmsg, "longitudeFirstGridpoint", msglons[first_j[0], first_i[0]]) 1502 setattr(newmsg, "nx", np.abs(first_i[0] - last_i[0])) 1503 setattr(newmsg, "ny", np.abs(first_j[0] - last_j[0])) 1504 1505 # Set *LastGridpoint attributes even if only used for gdtn=[0, 1, 40]. 1506 # This information is used to subset xarray datasets and even though 1507 # unnecessary for some supported grid types, it won't affect a grib2io 1508 # message to set them. 1509 setattr(newmsg, "latitudeLastGridpoint", msglats[last_j[0], last_i[0]]) 1510 setattr(newmsg, "longitudeLastGridpoint", msglons[last_j[0], last_i[0]]) 1511 1512 setattr( 1513 newmsg, 1514 "data", 1515 self.data[ 1516 min(first_j[0], last_j[0]) : max(first_j[0], last_j[0]), 1517 min(first_i[0], last_i[0]) : max(first_i[0], last_i[0]), 1518 ].copy(), 1519 ) 1520 1521 # Need to set the newmsg._sha1_section3 to a blank string so the grid 1522 # method ignores the cached lat/lon values. This will force the grid 1523 # method to recompute the lat/lon values for the subsetted grid. 1524 newmsg._sha1_section3 = "" 1525 newmsg.grid() 1526 1527 return newmsg 1528 1529 def validate(self): 1530 """ 1531 Validate a complete GRIB2 message. 1532 1533 The g2c library does its own internal validation when g2_gribend() is called, but 1534 we will check in grib2io also. The validation checks if the first 4 bytes in 1535 self._msg is 'GRIB' and '7777' as the last 4 bytes and that the message length in 1536 section 0 equals the length of the packed message. 1537 1538 Returns 1539 ------- 1540 `True` if the packed GRIB2 message is complete and well-formed, `False` otherwise. 1541 """ 1542 valid = False 1543 if hasattr(self,'_msg'): 1544 if self._msg[0:4]+self._msg[-4:] == b'GRIB7777': 1545 if self.section0[-1] == len(self._msg): 1546 valid = True 1547 return valid
GRIB2 Message base class.
GRIB2 Section 1, Identification Section
GRIB2 Section 3, Grid Definition Section
Product Definition Template
794 @property 795 def gdtn(self): 796 """Return Grid Definition Template Number""" 797 return self.section3[4]
Return Grid Definition Template Number
800 @property 801 def gdt(self): 802 """Return Grid Definition Template.""" 803 return self.gridDefinitionTemplate
Return Grid Definition Template.
806 @property 807 def pdtn(self): 808 """Return Product Definition Template Number.""" 809 return self.section4[1]
Return Product Definition Template Number.
812 @property 813 def pdt(self): 814 """Return Product Definition Template.""" 815 return self.productDefinitionTemplate
Return Product Definition Template.
818 @property 819 def drtn(self): 820 """Return Data Representation Template Number.""" 821 return self.section5[1]
Return Data Representation Template Number.
824 @property 825 def drt(self): 826 """Return Data Representation Template.""" 827 return self.dataRepresentationTemplate
Return Data Representation Template.
830 @property 831 def pdy(self): 832 """Return the PDY ('YYYYMMDD').""" 833 return ''.join([str(i) for i in self.section1[5:8]])
Return the PDY ('YYYYMMDD').
836 @property 837 def griddef(self): 838 """Return a Grib2GridDef instance for a GRIB2 message.""" 839 return Grib2GridDef.from_section3(self.section3)
Return a Grib2GridDef instance for a GRIB2 message.
853 @property 854 def min(self): 855 """Return minimum value of data.""" 856 return np.nanmin(self.data)
Return minimum value of data.
859 @property 860 def max(self): 861 """Return maximum value of data.""" 862 return np.nanmax(self.data)
Return maximum value of data.
865 @property 866 def mean(self): 867 """Return mean value of data.""" 868 return np.nanmean(self.data)
Return mean value of data.
871 @property 872 def median(self): 873 """Return median value of data.""" 874 return np.nanmedian(self.data)
Return median value of data.
921 def attrs_by_section(self, sect: int, values: bool=False): 922 """ 923 Provide a tuple of attribute names for the given GRIB2 section. 924 925 Parameters 926 ---------- 927 sect 928 The GRIB2 section number. 929 values 930 Optional (default is `False`) argument to return attributes values. 931 932 Returns 933 ------- 934 attrs_by_section 935 A list of attribute names or dict of name:value pairs if `values = 936 True`. 937 """ 938 if sect in {0,1,6}: 939 attrs = templates._section_attrs[sect] 940 elif sect in {3,4,5}: 941 def _find_class_index(n): 942 _key = {3:'Grid', 4:'Product', 5:'Data'} 943 for i,c in enumerate(self.__class__.__mro__): 944 if _key[n] in c.__name__: 945 return i 946 else: 947 return [] 948 if sys.version_info.minor <= 8: 949 attrs = templates._section_attrs[sect]+\ 950 [a for a in dir(self.__class__.__mro__[_find_class_index(sect)]) if not a.startswith('_')] 951 else: 952 attrs = templates._section_attrs[sect]+\ 953 self.__class__.__mro__[_find_class_index(sect)]._attrs() 954 else: 955 attrs = [] 956 if values: 957 return {k:getattr(self,k) for k in attrs} 958 else: 959 return attrs
Provide a tuple of attribute names for the given GRIB2 section.
Parameters
- sect: The GRIB2 section number.
- values: Optional (default is
False
) argument to return attributes values.
Returns
- attrs_by_section: A list of attribute names or dict of name:value pairs if
values = True
.
962 def pack(self): 963 """ 964 Pack GRIB2 section data into a binary message. 965 966 It is the user's responsibility to populate the GRIB2 section 967 information with appropriate metadata. 968 """ 969 # Create beginning of packed binary message with section 0 and 1 data. 970 self._sections = [] 971 self._msg,self._pos = g2clib.grib2_create(self.indicatorSection[2:4],self.identificationSection) 972 self._sections += [0,1] 973 974 # Add section 2 if present. 975 if isinstance(self.section2,bytes) and len(self.section2) > 0: 976 self._msg,self._pos = g2clib.grib2_addlocal(self._msg,self.section2) 977 self._sections.append(2) 978 979 # Add section 3. 980 self.section3[1] = self.nx * self.ny 981 self._msg,self._pos = g2clib.grib2_addgrid(self._msg,self.gridDefinitionSection, 982 self.gridDefinitionTemplate, 983 self._deflist) 984 self._sections.append(3) 985 986 # Prepare data. 987 if self._data is None: 988 if self._ondiskarray is None: 989 raise ValueError("Grib2Message object has no data, thus it cannot be packed.") 990 field = np.copy(self.data) 991 if self.scanModeFlags is not None: 992 if self.scanModeFlags[3]: 993 fieldsave = field.astype('f') # Casting makes a copy 994 field[1::2,:] = fieldsave[1::2,::-1] 995 fld = field.astype('f') 996 997 # Prepare bitmap, if necessary 998 bitmapflag = self.bitMapFlag.value 999 if bitmapflag == 0: 1000 if self.bitmap is not None: 1001 bmap = np.ravel(self.bitmap).astype(DEFAULT_NUMPY_INT) 1002 else: 1003 bmap = np.ravel(np.where(np.isnan(fld),0,1)).astype(DEFAULT_NUMPY_INT) 1004 else: 1005 bmap = None 1006 1007 # Prepare data for packing if nans are present 1008 fld = np.ravel(fld) 1009 if bitmapflag in {0,254}: 1010 fld = np.where(np.isnan(fld),0,fld) 1011 else: 1012 if np.isnan(fld).any(): 1013 if hasattr(self,'priMissingValue'): 1014 fld = np.where(np.isnan(fld),self.priMissingValue,fld) 1015 if hasattr(self,'_missvalmap'): 1016 if hasattr(self,'priMissingValue'): 1017 fld = np.where(self._missvalmap==1,self.priMissingValue,fld) 1018 if hasattr(self,'secMissingValue'): 1019 fld = np.where(self._missvalmap==2,self.secMissingValue,fld) 1020 1021 # Add sections 4, 5, 6, and 7. 1022 self._msg,self._pos = g2clib.grib2_addfield(self._msg,self.pdtn, 1023 self.productDefinitionTemplate, 1024 self._coordlist, 1025 self.drtn, 1026 self.dataRepresentationTemplate, 1027 fld, 1028 bitmapflag, 1029 bmap) 1030 self._sections.append(4) 1031 self._sections.append(5) 1032 self._sections.append(6) 1033 self._sections.append(7) 1034 1035 # Finalize GRIB2 message with section 8. 1036 self._msg, self._pos = g2clib.grib2_end(self._msg) 1037 self._sections.append(8) 1038 self.section0[-1] = len(self._msg)
Pack GRIB2 section data into a binary message.
It is the user's responsibility to populate the GRIB2 section information with appropriate metadata.
1041 @property 1042 def data(self) -> np.array: 1043 """Access the unpacked data values.""" 1044 if self._data is None: 1045 if self._auto_nans != _AUTO_NANS: 1046 self._data = self._ondiskarray 1047 self._data = np.asarray(self._ondiskarray) 1048 return self._data
Access the unpacked data values.
1058 def flush_data(self): 1059 """ 1060 Flush the unpacked data values from the Grib2Message object. 1061 1062 Note: If the Grib2Message object was constructed from "scratch" (i.e. 1063 not read from file), this method will remove the data array from 1064 the object and it cannot be recovered. 1065 """ 1066 self._data = None 1067 self.bitmap = None
Flush the unpacked data values from the Grib2Message object.
Note: If the Grib2Message object was constructed from "scratch" (i.e. not read from file), this method will remove the data array from the object and it cannot be recovered.
1078 def latlons(self, *args, **kwrgs): 1079 """Alias for `grib2io.Grib2Message.grid` method.""" 1080 return self.grid(*args, **kwrgs)
Alias for grib2io.Grib2Message.grid
method.
1083 def grid(self, unrotate: bool=True): 1084 """ 1085 Return lats,lons (in degrees) of grid. 1086 1087 Currently can handle reg. lat/lon,cglobal Gaussian, mercator, 1088 stereographic, lambert conformal, albers equal-area, space-view and 1089 azimuthal equidistant grids. 1090 1091 Parameters 1092 ---------- 1093 unrotate 1094 If `True` [DEFAULT], and grid is rotated lat/lon, then unrotate the 1095 grid, otherwise `False`, do not. 1096 1097 Returns 1098 ------- 1099 lats, lons : numpy.ndarray 1100 Returns two numpy.ndarrays with dtype=numpy.float32 of grid 1101 latitudes and longitudes in units of degrees. 1102 """ 1103 if self._sha1_section3 in _latlon_datastore.keys(): 1104 return (_latlon_datastore[self._sha1_section3]['latitude'], 1105 _latlon_datastore[self._sha1_section3]['longitude']) 1106 gdtn = self.gridDefinitionTemplateNumber.value 1107 gdtmpl = self.gridDefinitionTemplate 1108 reggrid = self.gridDefinitionSection[2] == 0 # This means regular 2-d grid 1109 if gdtn == 0: 1110 # Regular lat/lon grid 1111 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1112 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1113 dlon = self.gridlengthXDirection 1114 dlat = self.gridlengthYDirection 1115 if lon2 < lon1 and dlon < 0: lon1 = -lon1 1116 lats = np.linspace(lat1,lat2,self.ny) 1117 if reggrid: 1118 lons = np.linspace(lon1,lon2,self.nx) 1119 else: 1120 lons = np.linspace(lon1,lon2,self.ny*2) 1121 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1122 elif gdtn == 1: # Rotated Lat/Lon grid 1123 pj = pyproj.Proj(self.projParameters) 1124 lat1,lon1 = self.latitudeFirstGridpoint,self.longitudeFirstGridpoint 1125 lat2,lon2 = self.latitudeLastGridpoint,self.longitudeLastGridpoint 1126 if lon1 > 180.0: lon1 -= 360.0 1127 if lon2 > 180.0: lon2 -= 360.0 1128 lats = np.linspace(lat1,lat2,self.ny) 1129 lons = np.linspace(lon1,lon2,self.nx) 1130 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1131 if unrotate: 1132 from grib2io.utils import rotated_grid 1133 lats,lons = rotated_grid.unrotate(lats,lons,self.anglePoleRotation, 1134 self.latitudeSouthernPole, 1135 self.longitudeSouthernPole) 1136 elif gdtn == 40: # Gaussian grid (only works for global!) 1137 from grib2io.utils.gauss_grid import gaussian_latitudes 1138 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1139 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1140 nlats = self.ny 1141 if not reggrid: # Reduced Gaussian grid. 1142 nlons = 2*nlats 1143 dlon = 360./nlons 1144 else: 1145 nlons = self.nx 1146 dlon = self.gridlengthXDirection 1147 lons = np.linspace(lon1,lon2,nlons) 1148 # Compute Gaussian lats (north to south) 1149 lats = gaussian_latitudes(nlats) 1150 if lat1 < lat2: # reverse them if necessary 1151 lats = lats[::-1] 1152 lons,lats = np.meshgrid(lons,lats) 1153 elif gdtn in {10,20,30,31,110}: 1154 # Mercator, Lambert Conformal, Stereographic, Albers Equal Area, 1155 # Azimuthal Equidistant 1156 dx,dy = self.gridlengthXDirection, self.gridlengthYDirection 1157 lon1,lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1158 pj = pyproj.Proj(self.projParameters) 1159 llcrnrx, llcrnry = pj(lon1,lat1) 1160 x = llcrnrx+dx*np.arange(self.nx) 1161 y = llcrnry+dy*np.arange(self.ny) 1162 x,y = np.meshgrid(x, y) 1163 lons,lats = pj(x, y, inverse=True) 1164 elif gdtn == 90: 1165 # Satellite Projection 1166 dx = self.gridlengthXDirection 1167 dy = self.gridlengthYDirection 1168 pj = pyproj.Proj(self.projParameters) 1169 x = dx*np.indices((self.ny,self.nx),'f')[1,:,:] 1170 x -= 0.5*x.max() 1171 y = dy*np.indices((self.ny,self.nx),'f')[0,:,:] 1172 y -= 0.5*y.max() 1173 lons,lats = pj(x,y,inverse=True) 1174 # Set lons,lats to 1.e30 where undefined 1175 abslons = np.fabs(lons) 1176 abslats = np.fabs(lats) 1177 lons = np.where(abslons < 1.e20, lons, 1.e30) 1178 lats = np.where(abslats < 1.e20, lats, 1.e30) 1179 elif gdtn == 32769: 1180 # Special NCEP Grid, Rotated Lat/Lon, Arakawa E-Grid (Non-Staggered) 1181 from grib2io.utils import arakawa_rotated_grid 1182 from grib2io.utils.rotated_grid import DEG2RAD 1183 di, dj = 0.0, 0.0 1184 do_180 = False 1185 idir = 1 if self.scanModeFlags[0] == 0 else -1 1186 jdir = -1 if self.scanModeFlags[1] == 0 else 1 1187 grid_oriented = 0 if self.resolutionAndComponentFlags[4] == 0 else 1 1188 do_rot = 1 if grid_oriented == 1 else 0 1189 la1 = self.latitudeFirstGridpoint 1190 lo1 = self.longitudeFirstGridpoint 1191 clon = self.longitudeCenterGridpoint 1192 clat = self.latitudeCenterGridpoint 1193 lasp = clat - 90.0 1194 losp = clon 1195 llat, llon = arakawa_rotated_grid.ll2rot(la1,lo1,lasp,losp) 1196 la2, lo2 = arakawa_rotated_grid.rot2ll(-llat,-llon,lasp,losp) 1197 rlat = -llat 1198 rlon = -llon 1199 if self.nx == 1: 1200 di = 0.0 1201 elif idir == 1: 1202 ti = rlon 1203 while ti < llon: 1204 ti += 360.0 1205 di = (ti - llon)/float(self.nx-1) 1206 else: 1207 ti = llon 1208 while ti < rlon: 1209 ti += 360.0 1210 di = (ti - rlon)/float(self.nx-1) 1211 if self.ny == 1: 1212 dj = 0.0 1213 else: 1214 dj = (rlat - llat)/float(self.ny-1) 1215 if dj < 0.0: 1216 dj = -dj 1217 if idir == 1: 1218 if llon > rlon: 1219 llon -= 360.0 1220 if llon < 0 and rlon > 0: 1221 do_180 = True 1222 else: 1223 if rlon > llon: 1224 rlon -= 360.0 1225 if rlon < 0 and llon > 0: 1226 do_180 = True 1227 xlat1d = llat + (np.arange(self.ny)*jdir*dj) 1228 xlon1d = llon + (np.arange(self.nx)*idir*di) 1229 xlons, xlats = np.meshgrid(xlon1d,xlat1d) 1230 rot2ll_vectorized = np.vectorize(arakawa_rotated_grid.rot2ll) 1231 lats, lons = rot2ll_vectorized(xlats,xlons,lasp,losp) 1232 if do_180: 1233 lons = np.where(lons>180.0,lons-360.0,lons) 1234 vector_rotation_angles_vectorized = np.vectorize(arakawa_rotated_grid.vector_rotation_angles) 1235 rots = vector_rotation_angles_vectorized(lats, lons, clat, losp, xlats) 1236 del xlat1d, xlon1d, xlats, xlons 1237 else: 1238 raise ValueError('Unsupported grid') 1239 1240 _latlon_datastore[self._sha1_section3] = dict(latitude=lats,longitude=lons) 1241 try: 1242 _latlon_datastore[self._sha1_section3]['vector_rotation_angles'] = rots 1243 except(NameError): 1244 pass 1245 1246 return lats, lons
Return lats,lons (in degrees) of grid.
Currently can handle reg. lat/lon,cglobal Gaussian, mercator, stereographic, lambert conformal, albers equal-area, space-view and azimuthal equidistant grids.
Parameters
- unrotate: If
True
[DEFAULT], and grid is rotated lat/lon, then unrotate the grid, otherwiseFalse
, do not.
Returns
- lats, lons (numpy.ndarray): Returns two numpy.ndarrays with dtype=numpy.float32 of grid latitudes and longitudes in units of degrees.
1249 def map_keys(self): 1250 """ 1251 Unpack data grid replacing integer values with strings. 1252 1253 These types of fields are categorical or classifications where data 1254 values do not represent an observable or predictable physical quantity. 1255 An example of such a field would be [Dominant Precipitation Type - 1256 DPTYPE](https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table4-201.shtml) 1257 1258 Returns 1259 ------- 1260 map_keys 1261 numpy.ndarray of string values per element. 1262 """ 1263 hold_auto_nans = _AUTO_NANS 1264 set_auto_nans(False) 1265 1266 if (np.all(self.section1[0:2] == [7, 14]) and self.shortName == "PWTHER") or ( 1267 self._isNDFD and self.shortName in {"WX", "WWA"} 1268 ): 1269 keys = utils.decode_wx_strings(self.section2) 1270 if hasattr(self,'priMissingValue') and self.priMissingValue not in [None,0]: 1271 keys[int(self.priMissingValue)] = 'Missing' 1272 if hasattr(self,'secMissingValue') and self.secMissingValue not in [None,0]: 1273 keys[int(self.secMissingValue)] = 'Missing' 1274 u,inv = np.unique(self.data,return_inverse=True) 1275 fld = np.array([keys[x] for x in u])[inv].reshape(self.data.shape) 1276 else: 1277 # For data whose units are defined in a code table (i.e. classification or mask) 1278 tblname = re.findall(r'\d\.\d+',self.units,re.IGNORECASE)[0] 1279 fld = self.data.astype(np.int32).astype(str) 1280 tbl = tables.get_table(tblname,expand=True) 1281 for val in np.unique(fld): 1282 fld = np.where(fld==val,tbl[val],fld) 1283 set_auto_nans(hold_auto_nans) 1284 return fld
Unpack data grid replacing integer values with strings.
These types of fields are categorical or classifications where data values do not represent an observable or predictable physical quantity. An example of such a field would be Dominant Precipitation Type - DPTYPE
Returns
- map_keys: numpy.ndarray of string values per element.
1287 def to_bytes(self, validate: bool=True): 1288 """ 1289 Return packed GRIB2 message in bytes format. 1290 1291 This will be useful for exporting data in non-file formats. For example, 1292 can be used to output grib data directly to S3 using the boto3 client 1293 without the need to write a temporary file to upload first. 1294 1295 Parameters 1296 ---------- 1297 validate: default=True 1298 If `True` (DEFAULT), validates first/last four bytes for proper 1299 formatting, else returns None. If `False`, message is output as is. 1300 1301 Returns 1302 ------- 1303 to_bytes 1304 Returns GRIB2 formatted message as bytes. 1305 """ 1306 if hasattr(self,'_msg'): 1307 if validate: 1308 if self.validate(): 1309 return self._msg 1310 else: 1311 return None 1312 else: 1313 return self._msg 1314 else: 1315 return None
Return packed GRIB2 message in bytes format.
This will be useful for exporting data in non-file formats. For example, can be used to output grib data directly to S3 using the boto3 client without the need to write a temporary file to upload first.
Parameters
- validate (default=True):
If
True
(DEFAULT), validates first/last four bytes for proper formatting, else returns None. IfFalse
, message is output as is.
Returns
- to_bytes: Returns GRIB2 formatted message as bytes.
1318 def interpolate(self, method, grid_def_out, method_options=None, drtn=None, 1319 num_threads=1): 1320 """ 1321 Grib2Message Interpolator 1322 1323 Performs spatial interpolation via the [NCEPLIBS-ip 1324 library](https://github.com/NOAA-EMC/NCEPLIBS-ip). This interpolate 1325 method only supports scalar interpolation. If you need to perform 1326 vector interpolation, use the module-level `grib2io.interpolate` 1327 function. 1328 1329 Parameters 1330 ---------- 1331 method 1332 Interpolate method to use. This can either be an integer or string 1333 using the following mapping: 1334 1335 | Interpolate Scheme | Integer Value | 1336 | :---: | :---: | 1337 | 'bilinear' | 0 | 1338 | 'bicubic' | 1 | 1339 | 'neighbor' | 2 | 1340 | 'budget' | 3 | 1341 | 'spectral' | 4 | 1342 | 'neighbor-budget' | 6 | 1343 1344 grid_def_out : grib2io.Grib2GridDef 1345 Grib2GridDef object of the output grid. 1346 method_options : list of ints, optional 1347 Interpolation options. See the NCEPLIBS-ip documentation for 1348 more information on how these are used. 1349 drtn 1350 Data Representation Template to be used for the returned 1351 interpolated GRIB2 message. When `None`, the data representation 1352 template of the source GRIB2 message is used. Once again, it is the 1353 user's responsibility to properly set the Data Representation 1354 Template attributes. 1355 num_threads : int, optional 1356 Number of OpenMP threads to use for interpolation. The default 1357 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1358 was not built with OpenMP, then this keyword argument and value 1359 will have no impact. 1360 1361 Returns 1362 ------- 1363 interpolate 1364 If interpolating to a grid, a new Grib2Message object is returned. 1365 The GRIB2 metadata of the new Grib2Message object is identical to 1366 the input except where required to be different because of the new 1367 grid specs and possibly a new data representation template. 1368 1369 If interpolating to station points, the interpolated data values are 1370 returned as a numpy.ndarray. 1371 """ 1372 section0 = self.section0 1373 section0[-1] = 0 1374 gds = [0, grid_def_out.npoints, 0, 255, grid_def_out.gdtn] 1375 section3 = np.concatenate((gds,grid_def_out.gdt)) 1376 drtn = self.drtn if drtn is None else drtn 1377 1378 msg = Grib2Message(section0,self.section1,self.section2,section3, 1379 self.section4,None,self.bitMapFlag.value,drtn=drtn) 1380 1381 msg._msgnum = -1 1382 msg._deflist = self._deflist 1383 msg._coordlist = self._coordlist 1384 shape = (msg.ny,msg.nx) 1385 ndim = 2 1386 if msg.typeOfValues == 0: 1387 dtype = 'float32' 1388 elif msg.typeOfValues == 1: 1389 dtype = 'int32' 1390 msg._data = interpolate(self.data,method,Grib2GridDef.from_section3(self.section3),grid_def_out, 1391 method_options=method_options,num_threads=num_threads).reshape(msg.ny,msg.nx) 1392 msg.section5[0] = grid_def_out.npoints 1393 return msg
Grib2Message Interpolator
Performs spatial interpolation via the NCEPLIBS-ip
library. This interpolate
method only supports scalar interpolation. If you need to perform
vector interpolation, use the module-level grib2io.interpolate
function.
Parameters
- method: Interpolate method to use. This can either be an integer or string using the following mapping:
Interpolate Scheme | Integer Value |
---|---|
'bilinear' | 0 |
'bicubic' | 1 |
'neighbor' | 2 |
'budget' | 3 |
'spectral' | 4 |
'neighbor-budget' | 6 |
- grid_def_out (grib2io.Grib2GridDef): Grib2GridDef object of the output grid.
- method_options (list of ints, optional): Interpolation options. See the NCEPLIBS-ip documentation for more information on how these are used.
- drtn: Data Representation Template to be used for the returned
interpolated GRIB2 message. When
None
, the data representation template of the source GRIB2 message is used. Once again, it is the user's responsibility to properly set the Data Representation Template attributes. - num_threads (int, optional): Number of OpenMP threads to use for interpolation. The default value is 1. If NCEPLIBS-ip and grib2io's iplib extension module was not built with OpenMP, then this keyword argument and value will have no impact.
Returns
- interpolate: If interpolating to a grid, a new Grib2Message object is returned. The GRIB2 metadata of the new Grib2Message object is identical to the input except where required to be different because of the new grid specs and possibly a new data representation template.
If interpolating to station points, the interpolated data values are returned as a numpy.ndarray.
1396 def subset(self, lats, lons): 1397 """ 1398 Return a spatial subset. 1399 1400 Currently only supports regular grids of the following types: 1401 1402 | Grid Type | gdtn | 1403 | :---: | :---: | 1404 | Latitude/Longitude, Equidistant Cylindrical, or Plate Carree | 0 | 1405 | Rotated Latitude/Longitude | 1 | 1406 | Mercator | 10 | 1407 | Polar Stereographic | 20 | 1408 | Lambert Conformal | 30 | 1409 | Albers Equal-Area | 31 | 1410 | Gaussian Latitude/Longitude | 40 | 1411 | Equatorial Azimuthal Equidistant Projection | 110 | 1412 1413 Parameters 1414 ---------- 1415 lats 1416 List or tuple of latitudes. The minimum and maximum latitudes will 1417 be used to define the southern and northern boundaries. 1418 1419 The order of the latitudes is not important. The function will 1420 determine which is the minimum and maximum. 1421 1422 The latitudes should be in decimal degrees with 0.0 at the equator, 1423 positive values in the northern hemisphere increasing to 90, and 1424 negative values in the southern hemisphere decreasing to -90. 1425 lons 1426 List or tuple of longitudes. The minimum and maximum longitudes 1427 will be used to define the western and eastern boundaries. 1428 1429 The order of the longitudes is not important. The function will 1430 determine which is the minimum and maximum. 1431 1432 GRIB2 longitudes should be in decimal degrees with 0.0 at the prime 1433 meridian, positive values increasing eastward to 360. There are no 1434 negative GRIB2 longitudes. 1435 1436 The typical west longitudes that start at 0.0 at the prime meridian 1437 and decrease to -180 westward, are converted to GRIB2 longitudes by 1438 '360 - (absolute value of the west longitude)' where typical 1439 eastern longitudes are unchanged as GRIB2 longitudes. 1440 1441 Returns 1442 ------- 1443 subset 1444 A spatial subset of a GRIB2 message. 1445 """ 1446 if self.gdtn not in [0, 1, 10, 20, 30, 31, 40, 110]: 1447 raise ValueError( 1448 """ 1449 1450Subset only works for 1451 Latitude/Longitude, Equidistant Cylindrical, or Plate Carree (gdtn=0) 1452 Rotated Latitude/Longitude (gdtn=1) 1453 Mercator (gdtn=10) 1454 Polar Stereographic (gdtn=20) 1455 Lambert Conformal (gdtn=30) 1456 Albers Equal-Area (gdtn=31) 1457 Gaussian Latitude/Longitude (gdtn=40) 1458 Equatorial Azimuthal Equidistant Projection (gdtn=110) 1459 1460""" 1461 ) 1462 1463 if self.nx == 0 or self.ny == 0: 1464 raise ValueError( 1465 """ 1466 1467Subset only works for regular grids. 1468 1469""" 1470 ) 1471 1472 newmsg = Grib2Message( 1473 np.copy(self.section0), 1474 np.copy(self.section1), 1475 np.copy(self.section2), 1476 np.copy(self.section3), 1477 np.copy(self.section4), 1478 np.copy(self.section5), 1479 ) 1480 1481 msglats, msglons = self.grid() 1482 1483 la1 = np.max(lats) 1484 lo1 = np.min(lons) 1485 la2 = np.min(lats) 1486 lo2 = np.max(lons) 1487 1488 # Find the indices of the first and last grid points to the nearest 1489 # lat/lon values in the grid. 1490 first_lat = np.abs(msglats - la1) 1491 first_lon = np.abs(msglons - lo1) 1492 max_idx = np.maximum(first_lat, first_lon) 1493 first_j, first_i = np.where(max_idx == np.min(max_idx)) 1494 1495 last_lat = np.abs(msglats - la2) 1496 last_lon = np.abs(msglons - lo2) 1497 max_idx = np.maximum(last_lat, last_lon) 1498 last_j, last_i = np.where(max_idx == np.min(max_idx)) 1499 1500 setattr(newmsg, "latitudeFirstGridpoint", msglats[first_j[0], first_i[0]]) 1501 setattr(newmsg, "longitudeFirstGridpoint", msglons[first_j[0], first_i[0]]) 1502 setattr(newmsg, "nx", np.abs(first_i[0] - last_i[0])) 1503 setattr(newmsg, "ny", np.abs(first_j[0] - last_j[0])) 1504 1505 # Set *LastGridpoint attributes even if only used for gdtn=[0, 1, 40]. 1506 # This information is used to subset xarray datasets and even though 1507 # unnecessary for some supported grid types, it won't affect a grib2io 1508 # message to set them. 1509 setattr(newmsg, "latitudeLastGridpoint", msglats[last_j[0], last_i[0]]) 1510 setattr(newmsg, "longitudeLastGridpoint", msglons[last_j[0], last_i[0]]) 1511 1512 setattr( 1513 newmsg, 1514 "data", 1515 self.data[ 1516 min(first_j[0], last_j[0]) : max(first_j[0], last_j[0]), 1517 min(first_i[0], last_i[0]) : max(first_i[0], last_i[0]), 1518 ].copy(), 1519 ) 1520 1521 # Need to set the newmsg._sha1_section3 to a blank string so the grid 1522 # method ignores the cached lat/lon values. This will force the grid 1523 # method to recompute the lat/lon values for the subsetted grid. 1524 newmsg._sha1_section3 = "" 1525 newmsg.grid() 1526 1527 return newmsg
Return a spatial subset.
Currently only supports regular grids of the following types:
Grid Type | gdtn |
---|---|
Latitude/Longitude, Equidistant Cylindrical, or Plate Carree | 0 |
Rotated Latitude/Longitude | 1 |
Mercator | 10 |
Polar Stereographic | 20 |
Lambert Conformal | 30 |
Albers Equal-Area | 31 |
Gaussian Latitude/Longitude | 40 |
Equatorial Azimuthal Equidistant Projection | 110 |
Parameters
- lats: List or tuple of latitudes. The minimum and maximum latitudes will be used to define the southern and northern boundaries.
The order of the latitudes is not important. The function will determine which is the minimum and maximum.
The latitudes should be in decimal degrees with 0.0 at the equator, positive values in the northern hemisphere increasing to 90, and negative values in the southern hemisphere decreasing to -90.
- lons: List or tuple of longitudes. The minimum and maximum longitudes will be used to define the western and eastern boundaries.
The order of the longitudes is not important. The function will determine which is the minimum and maximum.
GRIB2 longitudes should be in decimal degrees with 0.0 at the prime meridian, positive values increasing eastward to 360. There are no negative GRIB2 longitudes.
The typical west longitudes that start at 0.0 at the prime meridian and decrease to -180 westward, are converted to GRIB2 longitudes by '360 - (absolute value of the west longitude)' where typical eastern longitudes are unchanged as GRIB2 longitudes.
Returns
- subset: A spatial subset of a GRIB2 message.
1529 def validate(self): 1530 """ 1531 Validate a complete GRIB2 message. 1532 1533 The g2c library does its own internal validation when g2_gribend() is called, but 1534 we will check in grib2io also. The validation checks if the first 4 bytes in 1535 self._msg is 'GRIB' and '7777' as the last 4 bytes and that the message length in 1536 section 0 equals the length of the packed message. 1537 1538 Returns 1539 ------- 1540 `True` if the packed GRIB2 message is complete and well-formed, `False` otherwise. 1541 """ 1542 valid = False 1543 if hasattr(self,'_msg'): 1544 if self._msg[0:4]+self._msg[-4:] == b'GRIB7777': 1545 if self.section0[-1] == len(self._msg): 1546 valid = True 1547 return valid
Validate a complete GRIB2 message.
The g2c library does its own internal validation when g2_gribend() is called, but we will check in grib2io also. The validation checks if the first 4 bytes in self._msg is 'GRIB' and '7777' as the last 4 bytes and that the message length in section 0 equals the length of the packed message.
Returns
True
if the packed GRIB2 message is complete and well-formed,False
otherwise.
2022@dataclass 2023class Grib2GridDef: 2024 """ 2025 Class for Grid Definition Template Number and Template as attributes. 2026 2027 This allows for cleaner looking code when passing these metadata around. 2028 For example, the `grib2io._Grib2Message.interpolate` method and 2029 `grib2io.interpolate` function accepts these objects. 2030 """ 2031 gdtn: int 2032 gdt: NDArray 2033 2034 @classmethod 2035 def from_section3(cls, section3): 2036 return cls(section3[4],section3[5:]) 2037 2038 @property 2039 def nx(self): 2040 """Number of grid points in x-direction.""" 2041 return int(self.gdt[7]) 2042 2043 @property 2044 def ny(self): 2045 """Number of grid points in y-direction.""" 2046 return int(self.gdt[8]) 2047 2048 @property 2049 def npoints(self): 2050 """Total number of grid points.""" 2051 return int(self.gdt[7] * self.gdt[8]) 2052 2053 @property 2054 def shape(self): 2055 """Shape of the grid.""" 2056 return (int(self.ny), int(self.nx)) 2057 2058 def to_section3(self): 2059 """Return a full GRIB2 section3 array.""" 2060 return np.array( 2061 [0, self.npoints, 0, 0, self.gdtn] + list(self.gdt) 2062 ).astype(np.int64)
Class for Grid Definition Template Number and Template as attributes.
This allows for cleaner looking code when passing these metadata around.
For example, the grib2io._Grib2Message.interpolate
method and
grib2io.interpolate
function accepts these objects.
2038 @property 2039 def nx(self): 2040 """Number of grid points in x-direction.""" 2041 return int(self.gdt[7])
Number of grid points in x-direction.
2043 @property 2044 def ny(self): 2045 """Number of grid points in y-direction.""" 2046 return int(self.gdt[8])
Number of grid points in y-direction.
2048 @property 2049 def npoints(self): 2050 """Total number of grid points.""" 2051 return int(self.gdt[7] * self.gdt[8])
Total number of grid points.