grib2io
Introduction
grib2io is a Python package that provides an interface to the NCEP GRIB2 C (g2c) library for the purpose of reading and writing WMO GRIdded Binary, Edition 2 (GRIB2) messages. A physical file can contain one or more GRIB2 messages.
GRIB2 file IO is performed directly in Python. The unpacking/packing of GRIB2 integer, coded metadata and data sections is performed by the g2c library functions via the g2clib Cython extension module. The decoding/encoding of GRIB2 metadata is translated into more descriptive, plain language metadata by providing a mapping of the integer coded metadata to the appropriate GRIB2 code tables. The NCEP GRIB2 code tables are a part of the grib2io package.
Interpolation
As of grib2io v2.4.0, spatial interpolation via NCEPLIPS-ip Fortran library is now a part of the grib2io package. The separate component package, grib2io-interp, has been deprecated. grib2io-interp provided interpolation via F2PY interface to NCEPLIBS-ip, which has become difficult since the removal of distutils from Python 3.12+.
NCEPLIBS-ip interpolation Fortran subroutines contain the BIND(C)
attribute which
provides an equivalent C-interface. grib2io now provides a Cython-based interface, iplib
,
to these Fortran subroutines via their C-interface. If NCEPLIBS-ip was built with
OpenMP support, iplib
will provide functions for getting and setting the number of
OpenMP threads.
Tutorials
The following Jupyter Notebooks are available as tutorials:
1from ._grib2io import * 2from ._grib2io import __doc__ 3from ._grib2io import _Grib2Message 4 5__all__ = ['open', 'show_config', 'interpolate', 'interpolate_to_stations', 6 'tables', 'templates', 'utils', 7 'Grib2Message', '_Grib2Message', 'Grib2GridDef'] 8 9try: 10 from . import __config__ 11 __version__ = __config__.grib2io_version 12 has_interpolation = __config__.has_interpolation 13 has_openmp_support = __config__.has_openmp_support 14 g2c_static = __config__.g2c_static 15 ip_static = __config__.ip_static 16 extra_objects = __config__.extra_objects 17except(ImportError): 18 pass 19 20from .g2clib import __version__ as __g2clib_version__ 21from .g2clib import _has_jpeg 22from .g2clib import _has_png 23from .g2clib import _has_aec 24 25has_jpeg_support = bool(_has_jpeg) 26has_png_support = bool(_has_png) 27has_aec_support = bool(_has_aec) 28 29from .tables.originating_centers import _ncep_grib2_table_version 30ncep_grib2_table_version = _ncep_grib2_table_version 31g2c_version = __g2clib_version__ 32 33def show_config(): 34 """Print grib2io build configuration information.""" 35 print(f'grib2io version {__version__} Configuration:') 36 print(f'') 37 print(f'NCEPLIBS-g2c library version: {__g2clib_version__}') 38 print(f'\tStatic library: {g2c_static}') 39 print(f'\tJPEG compression support: {has_jpeg_support}') 40 print(f'\tPNG compression support: {has_png_support}') 41 print(f'\tAEC compression support: {has_aec_support}') 42 print(f'') 43 print(f'NCEPLIPS-ip support: {has_interpolation}') 44 print(f'\tStatic library: {ip_static}') 45 print(f'\tOpenMP support: {has_openmp_support}') 46 print(f'') 47 print(f'Static libs:') 48 for lib in extra_objects: 49 print(f'\t{lib}') 50 print(f'') 51 print(f'NCEP GRIB2 Table Version: {_ncep_grib2_table_version}')
76class open(): 77 """ 78 GRIB2 File Object. 79 80 A physical file can contain one or more GRIB2 messages. When instantiated, 81 class `grib2io.open`, the file named `filename` is opened for reading (`mode 82 = 'r'`) and is automatically indexed. The indexing procedure reads some of 83 the GRIB2 metadata for all GRIB2 Messages. A GRIB2 Message may contain 84 submessages whereby Section 2-7 can be repeated. grib2io accommodates for 85 this by flattening any GRIB2 submessages into multiple individual messages. 86 87 It is important to note that GRIB2 files from some Meteorological agencies 88 contain other data than GRIB2 messages. GRIB2 files from ECMWF can contain 89 GRIB1 and GRIB2 messages. grib2io checks for these and safely ignores them. 90 91 Attributes 92 ---------- 93 closed : bool 94 `True` is file handle is close; `False` otherwise. 95 current_message : int 96 Current position of the file in units of GRIB2 Messages. 97 levels : tuple 98 Tuple containing a unique list of wgrib2-formatted level/layer strings. 99 messages : int 100 Count of GRIB2 Messages contained in the file. 101 mode : str 102 File IO mode of opening the file. 103 name : str 104 Full path name of the GRIB2 file. 105 size : int 106 Size of the file in units of bytes. 107 variables : tuple 108 Tuple containing a unique list of variable short names (i.e. GRIB2 109 abbreviation names). 110 """ 111 112 __slots__ = ('_fileid', '_filehandle', '_hasindex', '_index', '_nodata', 113 '_pos', 'closed', 'current_message', 'messages', 'mode', 114 'name', 'size') 115 116 def __init__(self, filename: str, mode: Literal["r", "w", "x"] = "r", **kwargs): 117 """ 118 Initialize GRIB2 File object instance. 119 120 Parameters 121 ---------- 122 filename 123 File name containing GRIB2 messages. 124 mode: default="r" 125 File access mode where "r" opens the files for reading only; "w" 126 opens the file for overwriting and "x" for writing to a new file. 127 """ 128 129 # Manage keywords 130 if "_xarray_backend" not in kwargs: 131 kwargs["_xarray_backend"] = False 132 self._nodata = False 133 else: 134 self._nodata = kwargs["_xarray_backend"] 135 136 # All write modes are read/write. 137 # All modes are binary. 138 if mode in ("a", "x", "w"): 139 mode += "+" 140 mode = mode + "b" 141 142 # Some GRIB2 files are gzipped, so check for that here, but 143 # raise error when using xarray backend. 144 if 'r' in mode: 145 self._filehandle = builtins.open(filename, mode=mode) 146 # Gzip files contain a 2-byte header b'\x1f\x8b'. 147 if self._filehandle.read(2) == b'\x1f\x8b': 148 self._filehandle.close() 149 if kwargs["_xarray_backend"]: 150 raise RuntimeError('Gzip GRIB2 files are not supported by the Xarray backend.') 151 import gzip 152 self._filehandle = gzip.open(filename, mode=mode) 153 else: 154 self._filehandle = builtins.open(filename, mode=mode) 155 else: 156 self._filehandle = builtins.open(filename, mode=mode) 157 158 self.name = os.path.abspath(filename) 159 fstat = os.stat(self.name) 160 self._hasindex = False 161 self._index = {} 162 self.mode = mode 163 self.messages = 0 164 self.current_message = 0 165 self.size = fstat.st_size 166 self.closed = self._filehandle.closed 167 self._fileid = hashlib.sha1((self.name+str(fstat.st_ino)+ 168 str(self.size)).encode('ASCII')).hexdigest() 169 if 'r' in self.mode: 170 self._build_index() 171 # FIX: Cannot perform reads on mode='a' 172 #if 'a' in self.mode and self.size > 0: self._build_index() 173 174 175 def __delete__(self, instance): 176 self.close() 177 del self._index 178 179 180 def __enter__(self): 181 return self 182 183 184 def __exit__(self, atype, value, traceback): 185 self.close() 186 187 188 def __iter__(self): 189 yield from self._index['msg'] 190 191 192 def __len__(self): 193 return self.messages 194 195 196 def __repr__(self): 197 strings = [] 198 for k in self.__slots__: 199 if k.startswith('_'): continue 200 strings.append('%s = %s\n'%(k,eval('self.'+k))) 201 return ''.join(strings) 202 203 204 def __getitem__(self, key): 205 if isinstance(key,int): 206 if abs(key) >= len(self._index['msg']): 207 raise IndexError("index out of range") 208 else: 209 return self._index['msg'][key] 210 elif isinstance(key,str): 211 return self.select(shortName=key) 212 elif isinstance(key,slice): 213 return self._index['msg'][key] 214 else: 215 raise KeyError('Key must be an integer, slice, or GRIB2 variable shortName.') 216 217 218 def _build_index(self): 219 """Perform indexing of GRIB2 Messages.""" 220 # Initialize index dictionary 221 if not self._hasindex: 222 self._index['sectionOffset'] = [] 223 self._index['sectionSize'] = [] 224 self._index['msgSize'] = [] 225 self._index['msgNumber'] = [] 226 self._index['msg'] = [] 227 self._index['isSubmessage'] = [] 228 self._hasindex = True 229 230 # Iterate 231 while True: 232 try: 233 # Initialize 234 bmapflag = None 235 pos = self._filehandle.tell() 236 section2 = b'' 237 trailer = b'' 238 _secpos = dict.fromkeys(range(8)) 239 _secsize = dict.fromkeys(range(8)) 240 _isSubmessage = False 241 242 # Ignore headers (usually text) that are not part of the GRIB2 243 # file. For example, NAVGEM files have a http header at the 244 # beginning that needs to be ignored. 245 246 # Read a byte at a time until "GRIB" is found. Using 247 # "wgrib2" on a NAVGEM file, the header was 421 bytes and 248 # decided to go to 2048 bytes to be safe. For normal GRIB2 249 # files this should be quick and break out of the first 250 # loop when "test_offset" is 0. 251 for test_offset in range(2048): 252 self._filehandle.seek(pos + test_offset) 253 header = struct.unpack(">I", self._filehandle.read(4))[0] 254 if header.to_bytes(4, "big") == b"GRIB": 255 pos = pos + test_offset 256 break 257 else: 258 # NOTE: Coming here means that no "GRIB" message identifier 259 # was found in the previous 2048 bytes. So here we continue 260 # the while True loop. 261 continue 262 263 # Read the rest of Section 0 using struct. 264 _secpos[0] = self._filehandle.tell()-4 265 _secsize[0] = 16 266 secmsg = self._filehandle.read(12) 267 section0 = np.concatenate(([header],list(struct.unpack('>HBBQ',secmsg))),dtype=np.int64) 268 269 # Make sure message is GRIB2. 270 if section0[3] != 2: 271 # Check for GRIB1 and ignore. 272 if secmsg[3] == 1: 273 warnings.warn("GRIB version 1 message detected. Ignoring...") 274 grib1_size = int.from_bytes(secmsg[:3],"big") 275 self._filehandle.seek(self._filehandle.tell()+grib1_size-16) 276 continue 277 else: 278 raise ValueError("Bad GRIB version number.") 279 280 # Read and unpack sections 1 through 8 which all follow a 281 # pattern of section size, number, and content. 282 while 1: 283 # Read first 5 bytes of the section which contains the size 284 # of the section (4 bytes) and the section number (1 byte). 285 secmsg = self._filehandle.read(5) 286 secsize, secnum = struct.unpack('>iB',secmsg) 287 288 # Record the offset of the section number and "append" the 289 # rest of the section to secmsg. 290 _secpos[secnum] = self._filehandle.tell()-5 291 _secsize[secnum] = secsize 292 if secnum in {1,3,4,5}: 293 secmsg += self._filehandle.read(secsize-5) 294 grbpos = 0 295 296 # Unpack section 297 if secnum == 1: 298 # Unpack Section 1 299 section1, grbpos = g2clib.unpack1(secmsg) 300 elif secnum == 2: 301 # Unpack Section 2 302 section2 = self._filehandle.read(secsize-5) 303 elif secnum == 3: 304 # Unpack Section 3 305 gds, gdt, deflist, grbpos = g2clib.unpack3(secmsg) 306 gds = gds.tolist() 307 gdt = gdt.tolist() 308 section3 = np.concatenate((gds,gdt)) 309 section3 = np.where(section3==4294967295,-1,section3) 310 elif secnum == 4: 311 # Unpack Section 4 312 pdtnum, pdt, coordlist, numcoord, grbpos = g2clib.unpack4(secmsg) 313 pdt = pdt.tolist() 314 section4 = np.concatenate((np.array((numcoord,pdtnum)),pdt)) 315 elif secnum == 5: 316 # Unpack Section 5 317 drtn, drt, npts, self._pos = g2clib.unpack5(secmsg) 318 section5 = np.concatenate((np.array((npts,drtn)),drt)) 319 section5 = np.where(section5==4294967295,-1,section5) 320 elif secnum == 6: 321 # Unpack Section 6 - Just the bitmap flag 322 bmapflag = struct.unpack('>B',self._filehandle.read(1))[0] 323 if bmapflag == 0: 324 bmappos = self._filehandle.tell()-6 325 elif bmapflag == 254: 326 # Do this to keep the previous position value 327 pass 328 else: 329 bmappos = None 330 self._filehandle.seek(self._filehandle.tell()+secsize-6) 331 elif secnum == 7: 332 # Do not unpack section 7 here, but move the file pointer 333 # to after section 7. 334 self._filehandle.seek(self._filehandle.tell()+secsize-5) 335 336 # Update the file index. 337 self.messages += 1 338 self._index['sectionOffset'].append(copy.deepcopy(_secpos)) 339 self._index['sectionSize'].append(copy.deepcopy(_secsize)) 340 self._index['msgSize'].append(section0[-1]) 341 self._index['msgNumber'].append(self.messages) 342 self._index['isSubmessage'].append(_isSubmessage) 343 344 # Create Grib2Message with data. 345 msg = Grib2Message(section0,section1,section2,section3,section4,section5,bmapflag) 346 msg._msgnum = self.messages-1 347 msg._deflist = deflist 348 msg._coordlist = coordlist 349 if not self._nodata: 350 msg._ondiskarray = Grib2MessageOnDiskArray((msg.ny,msg.nx), 2, 351 TYPE_OF_VALUES_DTYPE[msg.typeOfValues], 352 self._filehandle, 353 msg, pos, _secpos[6], _secpos[7]) 354 self._index['msg'].append(msg) 355 356 # If here, then we have moved through GRIB2 section 1-7. 357 # Now we need to check the next 4 bytes after section 7. 358 trailer = struct.unpack('>i',self._filehandle.read(4))[0] 359 360 # If we reach the GRIB2 trailer string ('7777'), then we 361 # can break begin processing the next GRIB2 message. If 362 # not, then we continue within the same iteration to 363 # process a GRIB2 submessage. 364 if trailer.to_bytes(4, "big") == b'7777': 365 break 366 else: 367 # If here, trailer should be the size of the first 368 # section of the next submessage, then the next byte 369 # is the section number. Check this value. 370 nextsec = struct.unpack('>B',self._filehandle.read(1))[0] 371 if nextsec not in {2,3,4}: 372 raise ValueError("Bad GRIB2 message structure.") 373 self._filehandle.seek(self._filehandle.tell()-5) 374 _isSubmessage = True 375 continue 376 else: 377 raise ValueError("Bad GRIB2 section number.") 378 379 except(struct.error): 380 if 'r' in self.mode: 381 self._filehandle.seek(0) 382 break 383 384 385 @property 386 def levels(self): 387 """Provides a unique tuple of level strings.""" 388 if self._hasindex and not self._nodata: 389 return tuple(sorted(set([msg.level for msg in self._index['msg']]))) 390 else: 391 return None 392 393 394 @property 395 def variables(self): 396 """Provides a unique tuple of variable shortName strings.""" 397 if self._hasindex and not self._nodata: 398 return tuple(sorted(set([msg.shortName for msg in self._index['msg']]))) 399 else: 400 return None 401 402 403 def close(self): 404 """Close the file handle.""" 405 if not self._filehandle.closed: 406 self.messages = 0 407 self.current_message = 0 408 self._filehandle.close() 409 self.closed = self._filehandle.closed 410 411 412 def read(self, size: Optional[int]=None): 413 """ 414 Read size amount of GRIB2 messages from the current position. 415 416 If no argument is given, then size is None and all messages are returned 417 from the current position in the file. This read method follows the 418 behavior of Python's builtin open() function, but whereas that operates 419 on units of bytes, we operate on units of GRIB2 messages. 420 421 Parameters 422 ---------- 423 size: default=None 424 The number of GRIB2 messages to read from the current position. If 425 no argument is give, the default value is None and remainder of 426 the file is read. 427 428 Returns 429 ------- 430 read 431 ``Grib2Message`` object when size = 1 or a list of Grib2Messages 432 when size > 1. 433 """ 434 if size is not None and size < 0: 435 size = None 436 if size is None or size > 1: 437 start = self.tell() 438 stop = self.messages if size is None else start+size 439 if size is None: 440 self.current_message = self.messages-1 441 else: 442 self.current_message += size 443 return self._index['msg'][slice(start,stop,1)] 444 elif size == 1: 445 self.current_message += 1 446 return self._index['msg'][self.current_message] 447 else: 448 None 449 450 451 def seek(self, pos: int): 452 """ 453 Set the position within the file in units of GRIB2 messages. 454 455 Parameters 456 ---------- 457 pos 458 The GRIB2 Message number to set the file pointer to. 459 """ 460 if self._hasindex: 461 self._filehandle.seek(self._index['sectionOffset'][0][pos]) 462 self.current_message = pos 463 464 465 def tell(self): 466 """Returns the position of the file in units of GRIB2 Messages.""" 467 return self.current_message 468 469 470 def select(self, **kwargs): 471 """Select GRIB2 messages by `Grib2Message` attributes.""" 472 # TODO: Added ability to process multiple values for each keyword (attribute) 473 idxs = [] 474 nkeys = len(kwargs.keys()) 475 for k,v in kwargs.items(): 476 for m in self._index['msg']: 477 if hasattr(m,k) and getattr(m,k) == v: idxs.append(m._msgnum) 478 idxs = np.array(idxs,dtype=np.int32) 479 return [self._index['msg'][i] for i in [ii[0] for ii in collections.Counter(idxs).most_common() if ii[1] == nkeys]] 480 481 482 def write(self, msg): 483 """ 484 Writes GRIB2 message object to file. 485 486 Parameters 487 ---------- 488 msg 489 GRIB2 message objects to write to file. 490 """ 491 if isinstance(msg,list): 492 for m in msg: 493 self.write(m) 494 return 495 496 if issubclass(msg.__class__,_Grib2Message): 497 if hasattr(msg,'_msg'): 498 self._filehandle.write(msg._msg) 499 else: 500 if msg._signature != msg._generate_signature(): 501 msg.pack() 502 self._filehandle.write(msg._msg) 503 else: 504 if hasattr(msg._data,'filehandle'): 505 msg._data.filehandle.seek(msg._data.offset) 506 self._filehandle.write(msg._data.filehandle.read(msg.section0[-1])) 507 else: 508 msg.pack() 509 self._filehandle.write(msg._msg) 510 self.flush() 511 self.size = os.path.getsize(self.name) 512 self._filehandle.seek(self.size-msg.section0[-1]) 513 self._build_index() 514 else: 515 raise TypeError("msg must be a Grib2Message object.") 516 return 517 518 519 def flush(self): 520 """Flush the file object buffer.""" 521 self._filehandle.flush() 522 523 524 def levels_by_var(self, name: str): 525 """ 526 Return a list of level strings given a variable shortName. 527 528 Parameters 529 ---------- 530 name 531 Grib2Message variable shortName 532 533 Returns 534 ------- 535 levels_by_var 536 A list of unique level strings. 537 """ 538 return list(sorted(set([msg.level for msg in self.select(shortName=name)]))) 539 540 541 def vars_by_level(self, level: str): 542 """ 543 Return a list of variable shortName strings given a level. 544 545 Parameters 546 ---------- 547 level 548 Grib2Message variable level 549 550 Returns 551 ------- 552 vars_by_level 553 A list of unique variable shortName strings. 554 """ 555 return list(sorted(set([msg.shortName for msg in self.select(level=level)])))
GRIB2 File Object.
A physical file can contain one or more GRIB2 messages. When instantiated,
class grib2io.open
, the file named filename
is opened for reading (mode
= 'r'
) and is automatically indexed. The indexing procedure reads some of
the GRIB2 metadata for all GRIB2 Messages. A GRIB2 Message may contain
submessages whereby Section 2-7 can be repeated. grib2io accommodates for
this by flattening any GRIB2 submessages into multiple individual messages.
It is important to note that GRIB2 files from some Meteorological agencies contain other data than GRIB2 messages. GRIB2 files from ECMWF can contain GRIB1 and GRIB2 messages. grib2io checks for these and safely ignores them.
Attributes
- closed (bool):
True
is file handle is close;False
otherwise. - current_message (int): Current position of the file in units of GRIB2 Messages.
- levels (tuple): Tuple containing a unique list of wgrib2-formatted level/layer strings.
- messages (int): Count of GRIB2 Messages contained in the file.
- mode (str): File IO mode of opening the file.
- name (str): Full path name of the GRIB2 file.
- size (int): Size of the file in units of bytes.
- variables (tuple): Tuple containing a unique list of variable short names (i.e. GRIB2 abbreviation names).
116 def __init__(self, filename: str, mode: Literal["r", "w", "x"] = "r", **kwargs): 117 """ 118 Initialize GRIB2 File object instance. 119 120 Parameters 121 ---------- 122 filename 123 File name containing GRIB2 messages. 124 mode: default="r" 125 File access mode where "r" opens the files for reading only; "w" 126 opens the file for overwriting and "x" for writing to a new file. 127 """ 128 129 # Manage keywords 130 if "_xarray_backend" not in kwargs: 131 kwargs["_xarray_backend"] = False 132 self._nodata = False 133 else: 134 self._nodata = kwargs["_xarray_backend"] 135 136 # All write modes are read/write. 137 # All modes are binary. 138 if mode in ("a", "x", "w"): 139 mode += "+" 140 mode = mode + "b" 141 142 # Some GRIB2 files are gzipped, so check for that here, but 143 # raise error when using xarray backend. 144 if 'r' in mode: 145 self._filehandle = builtins.open(filename, mode=mode) 146 # Gzip files contain a 2-byte header b'\x1f\x8b'. 147 if self._filehandle.read(2) == b'\x1f\x8b': 148 self._filehandle.close() 149 if kwargs["_xarray_backend"]: 150 raise RuntimeError('Gzip GRIB2 files are not supported by the Xarray backend.') 151 import gzip 152 self._filehandle = gzip.open(filename, mode=mode) 153 else: 154 self._filehandle = builtins.open(filename, mode=mode) 155 else: 156 self._filehandle = builtins.open(filename, mode=mode) 157 158 self.name = os.path.abspath(filename) 159 fstat = os.stat(self.name) 160 self._hasindex = False 161 self._index = {} 162 self.mode = mode 163 self.messages = 0 164 self.current_message = 0 165 self.size = fstat.st_size 166 self.closed = self._filehandle.closed 167 self._fileid = hashlib.sha1((self.name+str(fstat.st_ino)+ 168 str(self.size)).encode('ASCII')).hexdigest() 169 if 'r' in self.mode: 170 self._build_index() 171 # FIX: Cannot perform reads on mode='a' 172 #if 'a' in self.mode and self.size > 0: self._build_index()
Initialize GRIB2 File object instance.
Parameters
- filename: File name containing GRIB2 messages.
- mode (default="r"): File access mode where "r" opens the files for reading only; "w" opens the file for overwriting and "x" for writing to a new file.
385 @property 386 def levels(self): 387 """Provides a unique tuple of level strings.""" 388 if self._hasindex and not self._nodata: 389 return tuple(sorted(set([msg.level for msg in self._index['msg']]))) 390 else: 391 return None
Provides a unique tuple of level strings.
394 @property 395 def variables(self): 396 """Provides a unique tuple of variable shortName strings.""" 397 if self._hasindex and not self._nodata: 398 return tuple(sorted(set([msg.shortName for msg in self._index['msg']]))) 399 else: 400 return None
Provides a unique tuple of variable shortName strings.
403 def close(self): 404 """Close the file handle.""" 405 if not self._filehandle.closed: 406 self.messages = 0 407 self.current_message = 0 408 self._filehandle.close() 409 self.closed = self._filehandle.closed
Close the file handle.
412 def read(self, size: Optional[int]=None): 413 """ 414 Read size amount of GRIB2 messages from the current position. 415 416 If no argument is given, then size is None and all messages are returned 417 from the current position in the file. This read method follows the 418 behavior of Python's builtin open() function, but whereas that operates 419 on units of bytes, we operate on units of GRIB2 messages. 420 421 Parameters 422 ---------- 423 size: default=None 424 The number of GRIB2 messages to read from the current position. If 425 no argument is give, the default value is None and remainder of 426 the file is read. 427 428 Returns 429 ------- 430 read 431 ``Grib2Message`` object when size = 1 or a list of Grib2Messages 432 when size > 1. 433 """ 434 if size is not None and size < 0: 435 size = None 436 if size is None or size > 1: 437 start = self.tell() 438 stop = self.messages if size is None else start+size 439 if size is None: 440 self.current_message = self.messages-1 441 else: 442 self.current_message += size 443 return self._index['msg'][slice(start,stop,1)] 444 elif size == 1: 445 self.current_message += 1 446 return self._index['msg'][self.current_message] 447 else: 448 None
Read size amount of GRIB2 messages from the current position.
If no argument is given, then size is None and all messages are returned from the current position in the file. This read method follows the behavior of Python's builtin open() function, but whereas that operates on units of bytes, we operate on units of GRIB2 messages.
Parameters
- size (default=None): The number of GRIB2 messages to read from the current position. If no argument is give, the default value is None and remainder of the file is read.
Returns
- read:
Grib2Message
object when size = 1 or a list of Grib2Messages when size > 1.
451 def seek(self, pos: int): 452 """ 453 Set the position within the file in units of GRIB2 messages. 454 455 Parameters 456 ---------- 457 pos 458 The GRIB2 Message number to set the file pointer to. 459 """ 460 if self._hasindex: 461 self._filehandle.seek(self._index['sectionOffset'][0][pos]) 462 self.current_message = pos
Set the position within the file in units of GRIB2 messages.
Parameters
- pos: The GRIB2 Message number to set the file pointer to.
465 def tell(self): 466 """Returns the position of the file in units of GRIB2 Messages.""" 467 return self.current_message
Returns the position of the file in units of GRIB2 Messages.
470 def select(self, **kwargs): 471 """Select GRIB2 messages by `Grib2Message` attributes.""" 472 # TODO: Added ability to process multiple values for each keyword (attribute) 473 idxs = [] 474 nkeys = len(kwargs.keys()) 475 for k,v in kwargs.items(): 476 for m in self._index['msg']: 477 if hasattr(m,k) and getattr(m,k) == v: idxs.append(m._msgnum) 478 idxs = np.array(idxs,dtype=np.int32) 479 return [self._index['msg'][i] for i in [ii[0] for ii in collections.Counter(idxs).most_common() if ii[1] == nkeys]]
Select GRIB2 messages by Grib2Message
attributes.
482 def write(self, msg): 483 """ 484 Writes GRIB2 message object to file. 485 486 Parameters 487 ---------- 488 msg 489 GRIB2 message objects to write to file. 490 """ 491 if isinstance(msg,list): 492 for m in msg: 493 self.write(m) 494 return 495 496 if issubclass(msg.__class__,_Grib2Message): 497 if hasattr(msg,'_msg'): 498 self._filehandle.write(msg._msg) 499 else: 500 if msg._signature != msg._generate_signature(): 501 msg.pack() 502 self._filehandle.write(msg._msg) 503 else: 504 if hasattr(msg._data,'filehandle'): 505 msg._data.filehandle.seek(msg._data.offset) 506 self._filehandle.write(msg._data.filehandle.read(msg.section0[-1])) 507 else: 508 msg.pack() 509 self._filehandle.write(msg._msg) 510 self.flush() 511 self.size = os.path.getsize(self.name) 512 self._filehandle.seek(self.size-msg.section0[-1]) 513 self._build_index() 514 else: 515 raise TypeError("msg must be a Grib2Message object.") 516 return
Writes GRIB2 message object to file.
Parameters
- msg: GRIB2 message objects to write to file.
524 def levels_by_var(self, name: str): 525 """ 526 Return a list of level strings given a variable shortName. 527 528 Parameters 529 ---------- 530 name 531 Grib2Message variable shortName 532 533 Returns 534 ------- 535 levels_by_var 536 A list of unique level strings. 537 """ 538 return list(sorted(set([msg.level for msg in self.select(shortName=name)])))
Return a list of level strings given a variable shortName.
Parameters
- name: Grib2Message variable shortName
Returns
- levels_by_var: A list of unique level strings.
541 def vars_by_level(self, level: str): 542 """ 543 Return a list of variable shortName strings given a level. 544 545 Parameters 546 ---------- 547 level 548 Grib2Message variable level 549 550 Returns 551 ------- 552 vars_by_level 553 A list of unique variable shortName strings. 554 """ 555 return list(sorted(set([msg.shortName for msg in self.select(level=level)])))
Return a list of variable shortName strings given a level.
Parameters
- level: Grib2Message variable level
Returns
- vars_by_level: A list of unique variable shortName strings.
34def show_config(): 35 """Print grib2io build configuration information.""" 36 print(f'grib2io version {__version__} Configuration:') 37 print(f'') 38 print(f'NCEPLIBS-g2c library version: {__g2clib_version__}') 39 print(f'\tStatic library: {g2c_static}') 40 print(f'\tJPEG compression support: {has_jpeg_support}') 41 print(f'\tPNG compression support: {has_png_support}') 42 print(f'\tAEC compression support: {has_aec_support}') 43 print(f'') 44 print(f'NCEPLIPS-ip support: {has_interpolation}') 45 print(f'\tStatic library: {ip_static}') 46 print(f'\tOpenMP support: {has_openmp_support}') 47 print(f'') 48 print(f'Static libs:') 49 for lib in extra_objects: 50 print(f'\t{lib}') 51 print(f'') 52 print(f'NCEP GRIB2 Table Version: {_ncep_grib2_table_version}')
Print grib2io build configuration information.
1668def interpolate(a, method: Union[int, str], grid_def_in, grid_def_out, 1669 method_options=None, num_threads=1): 1670 """ 1671 This is the module-level interpolation function. 1672 1673 This interfaces with the [NCEPLIBS-ip library](https://github.com/NOAA-EMC/NCEPLIBS-ip) 1674 through grib2io's internal iplib Cython extension module. 1675 1676 Parameters 1677 ---------- 1678 a : numpy.ndarray or tuple 1679 Input data. If `a` is a `numpy.ndarray`, scalar interpolation will be 1680 performed. If `a` is a `tuple`, then vector interpolation will be 1681 performed with the assumption that u = a[0] and v = a[1] and are both 1682 `numpy.ndarray`. 1683 1684 These data are expected to be in 2-dimensional form with shape (ny, nx) 1685 or 3-dimensional (:, ny, nx) where the 1st dimension represents another 1686 spatial, temporal, or classification (i.e. ensemble members) dimension. 1687 The function will properly flatten the (ny,nx) dimensions into (nx * ny) 1688 acceptable for input into the interpolation subroutines. 1689 method 1690 Interpolate method to use. This can either be an integer or string using 1691 the following mapping: 1692 1693 | Interpolate Scheme | Integer Value | 1694 | :---: | :---: | 1695 | 'bilinear' | 0 | 1696 | 'bicubic' | 1 | 1697 | 'neighbor' | 2 | 1698 | 'budget' | 3 | 1699 | 'spectral' | 4 | 1700 | 'neighbor-budget' | 6 | 1701 1702 grid_def_in : grib2io.Grib2GridDef 1703 Grib2GridDef object for the input grid. 1704 grid_def_out : grib2io.Grib2GridDef 1705 Grib2GridDef object for the output grid or station points. 1706 method_options : list of ints, optional 1707 Interpolation options. See the NCEPLIBS-ip documentation for 1708 more information on how these are used. 1709 num_threads : int, optional 1710 Number of OpenMP threads to use for interpolation. The default 1711 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1712 was not built with OpenMP, then this keyword argument and value 1713 will have no impact. 1714 1715 Returns 1716 ------- 1717 interpolate 1718 Returns a `numpy.ndarray` when scalar interpolation is performed or a 1719 `tuple` of `numpy.ndarray`s when vector interpolation is performed with 1720 the assumptions that 0-index is the interpolated u and 1-index is the 1721 interpolated v. 1722 """ 1723 1724 from . import iplib 1725 1726 prev_num_threads = 1 1727 try: 1728 prev_num_threads = iplib.get_openmp_threads() 1729 iplib.set_openmp_threads(num_threads) 1730 except(AttributeError): 1731 pass 1732 1733 if isinstance(method,int) and method not in _interp_schemes.values(): 1734 raise ValueError('Invalid interpolation method.') 1735 elif isinstance(method,str): 1736 if method in _interp_schemes.keys(): 1737 method = _interp_schemes[method] 1738 else: 1739 raise ValueError('Invalid interpolation method.') 1740 1741 if method_options is None: 1742 method_options = np.zeros((20),dtype=np.int32) 1743 if method in {3,6}: 1744 method_options[0:2] = -1 1745 1746 km = 1 1747 mi = grid_def_in.npoints 1748 mo = grid_def_out.npoints 1749 1750 # Adjust shape of input array(s) 1751 a,newshp = _adjust_array_shape_for_interp(a,grid_def_in,grid_def_out) 1752 1753 # Call interpolation subroutines according to type of a. 1754 if isinstance(a,np.ndarray): 1755 # Scalar 1756 if np.any(np.isnan(a)): 1757 ibi = np.ones((km), dtype=np.int32) 1758 li = np.where(np.isnan(a),0,1).astype(np.uint8) 1759 else: 1760 ibi = np.zeros((km), dtype=np.int32) 1761 li = np.zeros(a.shape,dtype=np.uint8) 1762 no,rlat,rlon,ibo,lo,go,iret = iplib.interpolate_scalar(method, method_options, 1763 grid_def_in.gdtn, np.array(grid_def_in.gdt, dtype=np.int32), 1764 grid_def_out.gdtn, np.array(grid_def_out.gdt, dtype=np.int32), 1765 mi, mo, km, ibi, li, a) 1766 out = np.where(lo==0,np.nan,go).reshape(newshp) 1767 elif isinstance(a,tuple): 1768 # Vector 1769 if np.any(np.isnan(a)): 1770 ibi = np.ones((km), dtype=np.int32) 1771 li = np.where(np.isnan(a),0,1).astype(np.uint8) 1772 else: 1773 ibi = np.zeros((km), dtype=np.int32) 1774 li = np.zeros(a[0].shape,dtype=np.uint8) 1775 no,rlat,rlon,crot,srot,ibo,lo,uo,vo,iret = iplib.interpolate_vector(method, method_options, 1776 grid_def_in.gdtn, np.array(grid_def_in.gdt, dtype=np.int32), 1777 grid_def_out.gdtn, np.array(grid_def_out.gdt, dtype=np.int32), 1778 mi, mo, km, ibi, li, a[0], a[1]) 1779 uo = np.where(lo==0,np.nan,uo).reshape(newshp) 1780 vo = np.where(lo==0,np.nan,vo).reshape(newshp) 1781 out = (uo,vo) 1782 1783 try: 1784 iplib.set_openmp_threads(prev_num_threads) 1785 except(AttributeError): 1786 pass 1787 1788 return out
This is the module-level interpolation function.
This interfaces with the NCEPLIBS-ip library through grib2io's internal iplib Cython extension module.
Parameters
a (numpy.ndarray or tuple): Input data. If
a
is anumpy.ndarray
, scalar interpolation will be performed. Ifa
is atuple
, then vector interpolation will be performed with the assumption that u = a[0] and v = a[1] and are bothnumpy.ndarray
.These data are expected to be in 2-dimensional form with shape (ny, nx) or 3-dimensional (:, ny, nx) where the 1st dimension represents another spatial, temporal, or classification (i.e. ensemble members) dimension. The function will properly flatten the (ny,nx) dimensions into (nx * ny) acceptable for input into the interpolation subroutines.
- method: Interpolate method to use. This can either be an integer or string using the following mapping:
Interpolate Scheme | Integer Value |
---|---|
'bilinear' | 0 |
'bicubic' | 1 |
'neighbor' | 2 |
'budget' | 3 |
'spectral' | 4 |
'neighbor-budget' | 6 |
- grid_def_in (grib2io.Grib2GridDef): Grib2GridDef object for the input grid.
- grid_def_out (grib2io.Grib2GridDef): Grib2GridDef object for the output grid or station points.
- method_options (list of ints, optional): Interpolation options. See the NCEPLIBS-ip documentation for more information on how these are used.
- num_threads (int, optional): Number of OpenMP threads to use for interpolation. The default value is 1. If NCEPLIBS-ip and grib2io's iplib extension module was not built with OpenMP, then this keyword argument and value will have no impact.
Returns
- interpolate: Returns a
numpy.ndarray
when scalar interpolation is performed or atuple
ofnumpy.ndarray
s when vector interpolation is performed with the assumptions that 0-index is the interpolated u and 1-index is the interpolated v.
1791def interpolate_to_stations(a, method, grid_def_in, lats, lons, 1792 method_options=None, num_threads=1): 1793 """ 1794 Module-level interpolation function for interpolation to stations. 1795 1796 Interfaces with the [NCEPLIBS-ip library](https://github.com/NOAA-EMC/NCEPLIBS-ip) 1797 via grib2io's iplib Cython exntension module. It supports scalar and 1798 vector interpolation according to the type of object a. 1799 1800 Parameters 1801 ---------- 1802 a : numpy.ndarray or tuple 1803 Input data. If `a` is a `numpy.ndarray`, scalar interpolation will be 1804 performed. If `a` is a `tuple`, then vector interpolation will be 1805 performed with the assumption that u = a[0] and v = a[1] and are both 1806 `numpy.ndarray`. 1807 1808 These data are expected to be in 2-dimensional form with shape (ny, nx) 1809 or 3-dimensional (:, ny, nx) where the 1st dimension represents another 1810 spatial, temporal, or classification (i.e. ensemble members) dimension. 1811 The function will properly flatten the (ny,nx) dimensions into (nx * ny) 1812 acceptable for input into the interpolation subroutines. 1813 method 1814 Interpolate method to use. This can either be an integer or string using 1815 the following mapping: 1816 1817 | Interpolate Scheme | Integer Value | 1818 | :---: | :---: | 1819 | 'bilinear' | 0 | 1820 | 'bicubic' | 1 | 1821 | 'neighbor' | 2 | 1822 | 'budget' | 3 | 1823 | 'spectral' | 4 | 1824 | 'neighbor-budget' | 6 | 1825 1826 grid_def_in : grib2io.Grib2GridDef 1827 Grib2GridDef object for the input grid. 1828 lats : numpy.ndarray or list 1829 Latitudes for station points 1830 lons : numpy.ndarray or list 1831 Longitudes for station points 1832 method_options : list of ints, optional 1833 Interpolation options. See the NCEPLIBS-ip documentation for 1834 more information on how these are used. 1835 num_threads : int, optional 1836 Number of OpenMP threads to use for interpolation. The default 1837 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1838 was not built with OpenMP, then this keyword argument and value 1839 will have no impact. 1840 1841 Returns 1842 ------- 1843 interpolate_to_stations 1844 Returns a `numpy.ndarray` when scalar interpolation is performed or a 1845 `tuple` of `numpy.ndarray`s when vector interpolation is performed with 1846 the assumptions that 0-index is the interpolated u and 1-index is the 1847 interpolated v. 1848 """ 1849 from . import iplib 1850 1851 prev_num_threads = 1 1852 try: 1853 prev_num_threads = iplib.get_openmp_threads() 1854 iplib.set_openmp_threads(num_threads) 1855 except(AttributeError): 1856 pass 1857 1858 if isinstance(method,int) and method not in _interp_schemes.values(): 1859 raise ValueError('Invalid interpolation method.') 1860 elif isinstance(method,str): 1861 if method in _interp_schemes.keys(): 1862 method = _interp_schemes[method] 1863 else: 1864 raise ValueError('Invalid interpolation method.') 1865 1866 if method_options is None: 1867 method_options = np.zeros((20),dtype=np.int32) 1868 if method in {3,6}: 1869 method_options[0:2] = -1 1870 1871 # Check lats and lons 1872 if isinstance(lats,list): 1873 nlats = len(lats) 1874 elif isinstance(lats,np.ndarray) and len(lats.shape) == 1: 1875 nlats = lats.shape[0] 1876 else: 1877 raise ValueError('Station latitudes must be a list or 1-D NumPy array.') 1878 if isinstance(lons,list): 1879 nlons = len(lons) 1880 elif isinstance(lons,np.ndarray) and len(lons.shape) == 1: 1881 nlons = lons.shape[0] 1882 else: 1883 raise ValueError('Station longitudes must be a list or 1-D NumPy array.') 1884 if nlats != nlons: 1885 raise ValueError('Station lats and lons must be same size.') 1886 1887 km = 1 1888 mi = grid_def_in.npoints 1889 mo = nlats 1890 1891 # Adjust shape of input array(s) 1892 a,newshp = _adjust_array_shape_for_interp_stations(a,grid_def_in,mo) 1893 1894 # Use gdtn = -1 for stations and an empty template array 1895 gdtn = -1 1896 gdt = np.zeros((200),dtype=np.int32) 1897 1898 # Call interpolation subroutines according to type of a. 1899 if isinstance(a,np.ndarray): 1900 # Scalar 1901 ibi = np.zeros((km), dtype=np.int32) 1902 li = np.zeros(a.shape,dtype=np.uint8) 1903 no,rlat,rlon,ibo,lo,go,iret = iplib.interpolate_scalar(method, method_options, 1904 grid_def_in.gdtn, np.array(grid_def_in.gdt, dtype=np.int32), 1905 gdtn, gdt, 1906 mi, mo, km, ibi, li, a, 1907 lats=np.array(lats, dtype=np.float32), 1908 lons=np.array(lons, dtype=np.float32)) 1909 out = go.reshape(newshp) 1910 elif isinstance(a,tuple): 1911 # Vector 1912 ibi = np.zeros((km), dtype=np.int32) 1913 li = np.zeros(a[0].shape,dtype=np.uint8) 1914 no,rlat,rlon,crot,srot,ibo,lo,uo,vo,iret = iplib.interpolate_vector(method, method_options, 1915 grid_def_in.gdtn, np.array(grid_def_in.gdt, dtype=np.int32), 1916 gdtn, gdt, 1917 mi, mo, km, ibi, li, a[0], a[1], 1918 lats=np.array(lats, dtype=np.float32), 1919 lons=np.array(lons, dtype=np.float32)) 1920 out = (uo.reshape(newshp),vo.reshape(newshp)) 1921 1922 try: 1923 iplib.set_openmp_threads(prev_num_threads) 1924 except(AttributeError): 1925 pass 1926 1927 return out
Module-level interpolation function for interpolation to stations.
Interfaces with the NCEPLIBS-ip library via grib2io's iplib Cython exntension module. It supports scalar and vector interpolation according to the type of object a.
Parameters
a (numpy.ndarray or tuple): Input data. If
a
is anumpy.ndarray
, scalar interpolation will be performed. Ifa
is atuple
, then vector interpolation will be performed with the assumption that u = a[0] and v = a[1] and are bothnumpy.ndarray
.These data are expected to be in 2-dimensional form with shape (ny, nx) or 3-dimensional (:, ny, nx) where the 1st dimension represents another spatial, temporal, or classification (i.e. ensemble members) dimension. The function will properly flatten the (ny,nx) dimensions into (nx * ny) acceptable for input into the interpolation subroutines.
- method: Interpolate method to use. This can either be an integer or string using the following mapping:
Interpolate Scheme | Integer Value |
---|---|
'bilinear' | 0 |
'bicubic' | 1 |
'neighbor' | 2 |
'budget' | 3 |
'spectral' | 4 |
'neighbor-budget' | 6 |
- grid_def_in (grib2io.Grib2GridDef): Grib2GridDef object for the input grid.
- lats (numpy.ndarray or list): Latitudes for station points
- lons (numpy.ndarray or list): Longitudes for station points
- method_options (list of ints, optional): Interpolation options. See the NCEPLIBS-ip documentation for more information on how these are used.
- num_threads (int, optional): Number of OpenMP threads to use for interpolation. The default value is 1. If NCEPLIBS-ip and grib2io's iplib extension module was not built with OpenMP, then this keyword argument and value will have no impact.
Returns
- interpolate_to_stations: Returns a
numpy.ndarray
when scalar interpolation is performed or atuple
ofnumpy.ndarray
s when vector interpolation is performed with the assumptions that 0-index is the interpolated u and 1-index is the interpolated v.
558class Grib2Message: 559 """ 560 Creation class for a GRIB2 message. 561 562 This class returns a dynamically-created Grib2Message object that 563 inherits from `_Grib2Message` and grid, product, data representation 564 template classes according to the template numbers for the respective 565 sections. If `section3`, `section4`, or `section5` are omitted, then 566 the appropriate keyword arguments for the template number `gdtn=`, 567 `pdtn=`, or `drtn=` must be provided. 568 569 Parameters 570 ---------- 571 section0 572 GRIB2 section 0 array. 573 section1 574 GRIB2 section 1 array. 575 section2 576 Local Use section data. 577 section3 578 GRIB2 section 3 array. 579 section4 580 GRIB2 section 4 array. 581 section5 582 GRIB2 section 5 array. 583 584 Returns 585 ------- 586 Msg 587 A dynamically-create Grib2Message object that inherits from 588 _Grib2Message, a grid definition template class, product 589 definition template class, and a data representation template 590 class. 591 """ 592 def __new__(self, section0: NDArray = np.array([struct.unpack('>I',b'GRIB')[0],0,0,2,0]), 593 section1: NDArray = np.zeros((13),dtype=np.int64), 594 section2: Optional[bytes] = None, 595 section3: Optional[NDArray] = None, 596 section4: Optional[NDArray] = None, 597 section5: Optional[NDArray] = None, *args, **kwargs): 598 599 if np.all(section1==0): 600 try: 601 # Python >= 3.10 602 section1[5:11] = datetime.datetime.fromtimestamp(0, datetime.UTC).timetuple()[:6] 603 except(AttributeError): 604 # Python < 3.10 605 section1[5:11] = datetime.datetime.utcfromtimestamp(0).timetuple()[:6] 606 607 bases = list() 608 if section3 is None: 609 if 'gdtn' in kwargs.keys(): 610 gdtn = kwargs['gdtn'] 611 Gdt = templates.gdt_class_by_gdtn(gdtn) 612 bases.append(Gdt) 613 section3 = np.zeros((Gdt._len+5),dtype=np.int64) 614 section3[4] = gdtn 615 else: 616 raise ValueError("Must provide GRIB2 Grid Definition Template Number or section 3 array") 617 else: 618 gdtn = section3[4] 619 Gdt = templates.gdt_class_by_gdtn(gdtn) 620 bases.append(Gdt) 621 622 if section4 is None: 623 if 'pdtn' in kwargs.keys(): 624 pdtn = kwargs['pdtn'] 625 Pdt = templates.pdt_class_by_pdtn(pdtn) 626 bases.append(Pdt) 627 section4 = np.zeros((Pdt._len+2),dtype=np.int64) 628 section4[1] = pdtn 629 else: 630 raise ValueError("Must provide GRIB2 Production Definition Template Number or section 4 array") 631 else: 632 pdtn = section4[1] 633 Pdt = templates.pdt_class_by_pdtn(pdtn) 634 bases.append(Pdt) 635 636 if section5 is None: 637 if 'drtn' in kwargs.keys(): 638 drtn = kwargs['drtn'] 639 Drt = templates.drt_class_by_drtn(drtn) 640 bases.append(Drt) 641 section5 = np.zeros((Drt._len+2),dtype=np.int64) 642 section5[1] = drtn 643 else: 644 raise ValueError("Must provide GRIB2 Data Representation Template Number or section 5 array") 645 else: 646 drtn = section5[1] 647 Drt = templates.drt_class_by_drtn(drtn) 648 bases.append(Drt) 649 650 # attempt to use existing Msg class if it has already been made with gdtn,pdtn,drtn combo 651 try: 652 Msg = _msg_class_store[f"{gdtn}:{pdtn}:{drtn}"] 653 except KeyError: 654 @dataclass(init=False, repr=False) 655 class Msg(_Grib2Message, *bases): 656 pass 657 _msg_class_store[f"{gdtn}:{pdtn}:{drtn}"] = Msg 658 659 return Msg(section0, section1, section2, section3, section4, section5, *args)
Creation class for a GRIB2 message.
This class returns a dynamically-created Grib2Message object that
inherits from _Grib2Message
and grid, product, data representation
template classes according to the template numbers for the respective
sections. If section3
, section4
, or section5
are omitted, then
the appropriate keyword arguments for the template number gdtn=
,
pdtn=
, or drtn=
must be provided.
Parameters
- section0: GRIB2 section 0 array.
- section1: GRIB2 section 1 array.
- section2: Local Use section data.
- section3: GRIB2 section 3 array.
- section4: GRIB2 section 4 array.
- section5: GRIB2 section 5 array.
Returns
- Msg: A dynamically-create Grib2Message object that inherits from _Grib2Message, a grid definition template class, product definition template class, and a data representation template class.
662@dataclass 663class _Grib2Message: 664 """ 665 GRIB2 Message base class. 666 """ 667 # GRIB2 Sections 668 section0: NDArray = field(init=True,repr=False) 669 section1: NDArray = field(init=True,repr=False) 670 section2: bytes = field(init=True,repr=False) 671 section3: NDArray = field(init=True,repr=False) 672 section4: NDArray = field(init=True,repr=False) 673 section5: NDArray = field(init=True,repr=False) 674 bitMapFlag: templates.Grib2Metadata = field(init=True,repr=False,default=255) 675 676 # Section 0 looked up attributes 677 indicatorSection: NDArray = field(init=False,repr=False,default=templates.IndicatorSection()) 678 discipline: templates.Grib2Metadata = field(init=False,repr=False,default=templates.Discipline()) 679 680 # Section 1 looked up attributes 681 identificationSection: NDArray = field(init=False,repr=False,default=templates.IdentificationSection()) 682 originatingCenter: templates.Grib2Metadata = field(init=False,repr=False,default=templates.OriginatingCenter()) 683 originatingSubCenter: templates.Grib2Metadata = field(init=False,repr=False,default=templates.OriginatingSubCenter()) 684 masterTableInfo: templates.Grib2Metadata = field(init=False,repr=False,default=templates.MasterTableInfo()) 685 localTableInfo: templates.Grib2Metadata = field(init=False,repr=False,default=templates.LocalTableInfo()) 686 significanceOfReferenceTime: templates.Grib2Metadata = field(init=False,repr=False,default=templates.SignificanceOfReferenceTime()) 687 year: int = field(init=False,repr=False,default=templates.Year()) 688 month: int = field(init=False,repr=False,default=templates.Month()) 689 day: int = field(init=False,repr=False,default=templates.Day()) 690 hour: int = field(init=False,repr=False,default=templates.Hour()) 691 minute: int = field(init=False,repr=False,default=templates.Minute()) 692 second: int = field(init=False,repr=False,default=templates.Second()) 693 refDate: datetime.datetime = field(init=False,repr=False,default=templates.RefDate()) 694 productionStatus: templates.Grib2Metadata = field(init=False,repr=False,default=templates.ProductionStatus()) 695 typeOfData: templates.Grib2Metadata = field(init=False,repr=False,default=templates.TypeOfData()) 696 697 # Section 3 looked up common attributes. Other looked up attributes are available according 698 # to the Grid Definition Template. 699 gridDefinitionSection: NDArray = field(init=False,repr=False,default=templates.GridDefinitionSection()) 700 sourceOfGridDefinition: int = field(init=False,repr=False,default=templates.SourceOfGridDefinition()) 701 numberOfDataPoints: int = field(init=False,repr=False,default=templates.NumberOfDataPoints()) 702 interpretationOfListOfNumbers: templates.Grib2Metadata = field(init=False,repr=False,default=templates.InterpretationOfListOfNumbers()) 703 gridDefinitionTemplateNumber: templates.Grib2Metadata = field(init=False,repr=False,default=templates.GridDefinitionTemplateNumber()) 704 gridDefinitionTemplate: list = field(init=False,repr=False,default=templates.GridDefinitionTemplate()) 705 _earthparams: dict = field(init=False,repr=False,default=templates.EarthParams()) 706 _dxsign: float = field(init=False,repr=False,default=templates.DxSign()) 707 _dysign: float = field(init=False,repr=False,default=templates.DySign()) 708 _llscalefactor: float = field(init=False,repr=False,default=templates.LLScaleFactor()) 709 _lldivisor: float = field(init=False,repr=False,default=templates.LLDivisor()) 710 _xydivisor: float = field(init=False,repr=False,default=templates.XYDivisor()) 711 shapeOfEarth: templates.Grib2Metadata = field(init=False,repr=False,default=templates.ShapeOfEarth()) 712 earthShape: str = field(init=False,repr=False,default=templates.EarthShape()) 713 earthRadius: float = field(init=False,repr=False,default=templates.EarthRadius()) 714 earthMajorAxis: float = field(init=False,repr=False,default=templates.EarthMajorAxis()) 715 earthMinorAxis: float = field(init=False,repr=False,default=templates.EarthMinorAxis()) 716 resolutionAndComponentFlags: list = field(init=False,repr=False,default=templates.ResolutionAndComponentFlags()) 717 ny: int = field(init=False,repr=False,default=templates.Ny()) 718 nx: int = field(init=False,repr=False,default=templates.Nx()) 719 scanModeFlags: list = field(init=False,repr=False,default=templates.ScanModeFlags()) 720 projParameters: dict = field(init=False,repr=False,default=templates.ProjParameters()) 721 722 # Section 4 723 productDefinitionTemplateNumber: templates.Grib2Metadata = field(init=False,repr=False,default=templates.ProductDefinitionTemplateNumber()) 724 productDefinitionTemplate: NDArray = field(init=False,repr=False,default=templates.ProductDefinitionTemplate()) 725 726 # Section 5 looked up common attributes. Other looked up attributes are 727 # available according to the Data Representation Template. 728 numberOfPackedValues: int = field(init=False,repr=False,default=templates.NumberOfPackedValues()) 729 dataRepresentationTemplateNumber: templates.Grib2Metadata = field(init=False,repr=False,default=templates.DataRepresentationTemplateNumber()) 730 dataRepresentationTemplate: list = field(init=False,repr=False,default=templates.DataRepresentationTemplate()) 731 typeOfValues: templates.Grib2Metadata = field(init=False,repr=False,default=templates.TypeOfValues()) 732 733 def __post_init__(self): 734 """Set some attributes after init.""" 735 self._auto_nans = _AUTO_NANS 736 self._coordlist = np.zeros((0), dtype=np.float32) 737 self._data = None 738 self._deflist = np.zeros((0), dtype=np.int64) 739 self._msgnum = -1 740 self._ondiskarray = None 741 self._orig_section5 = np.copy(self.section5) 742 self._signature = self._generate_signature() 743 try: 744 self._sha1_section3 = hashlib.sha1(self.section3).hexdigest() 745 except(TypeError): 746 pass 747 self.bitMapFlag = templates.Grib2Metadata(self.bitMapFlag,table='6.0') 748 self.bitmap = None 749 750 @property 751 def _isNDFD(self): 752 """Check if GRIB2 message is from NWS NDFD""" 753 return np.all(self.section1[0:2]==[8,65535]) 754 755 @property 756 def _isAerosol(self): 757 """Check if GRIB2 message contains aerosol data""" 758 is_aero_template = self.productDefinitionTemplateNumber.value in tables.AEROSOL_PDTNS 759 is_aero_param = ((str(self.parameterCategory) == '13') | 760 (str(self.parameterCategory) == '20')) and str(self.parameterNumber) in tables.AEROSOL_PARAMS 761 # Check table 4.205 aerosol presence 762 is_aero_type = (str(self.parameterCategory) == '205' and 763 str(self.parameterNumber) == '1') 764 return is_aero_template or is_aero_param or is_aero_type 765 766 @property 767 def gdtn(self): 768 """Return Grid Definition Template Number""" 769 return self.section3[4] 770 771 772 @property 773 def gdt(self): 774 """Return Grid Definition Template.""" 775 return self.gridDefinitionTemplate 776 777 778 @property 779 def pdtn(self): 780 """Return Product Definition Template Number.""" 781 return self.section4[1] 782 783 784 @property 785 def pdt(self): 786 """Return Product Definition Template.""" 787 return self.productDefinitionTemplate 788 789 790 @property 791 def drtn(self): 792 """Return Data Representation Template Number.""" 793 return self.section5[1] 794 795 796 @property 797 def drt(self): 798 """Return Data Representation Template.""" 799 return self.dataRepresentationTemplate 800 801 802 @property 803 def pdy(self): 804 """Return the PDY ('YYYYMMDD').""" 805 return ''.join([str(i) for i in self.section1[5:8]]) 806 807 808 @property 809 def griddef(self): 810 """Return a Grib2GridDef instance for a GRIB2 message.""" 811 return Grib2GridDef.from_section3(self.section3) 812 813 814 @property 815 def lats(self): 816 """Return grid latitudes.""" 817 return self.latlons()[0] 818 819 820 @property 821 def lons(self): 822 """Return grid longitudes.""" 823 return self.latlons()[1] 824 825 @property 826 def min(self): 827 """Return minimum value of data.""" 828 return np.nanmin(self.data) 829 830 831 @property 832 def max(self): 833 """Return maximum value of data.""" 834 return np.nanmax(self.data) 835 836 837 @property 838 def mean(self): 839 """Return mean value of data.""" 840 return np.nanmean(self.data) 841 842 843 @property 844 def median(self): 845 """Return median value of data.""" 846 return np.nanmedian(self.data) 847 848 849 @property 850 def shape(self): 851 """Return shape of data.""" 852 return self.griddef.shape 853 854 855 def __repr__(self): 856 """ 857 Return an unambiguous string representation of the object. 858 859 Returns 860 ------- 861 repr 862 A string representation of the object, including information from 863 sections 0, 1, 3, 4, 5, and 6. 864 """ 865 info = '' 866 for sect in [0,1,3,4,5,6]: 867 for k,v in self.attrs_by_section(sect,values=True).items(): 868 info += f'Section {sect}: {k} = {v}\n' 869 return info 870 871 def __str__(self): 872 """ 873 Return a readable string representation of the object. 874 875 Returns 876 ------- 877 str 878 A formatted string representation of the object, including 879 selected attributes. 880 """ 881 return (f'{self._msgnum}:d={self.refDate}:{self.shortName}:' 882 f'{self.fullName} ({self.units}):{self.level}:' 883 f'{self.leadTime+self.duration}') 884 885 886 def _generate_signature(self): 887 """Generature SHA-1 hash string from GRIB2 integer sections.""" 888 return hashlib.sha1(np.concatenate((self.section0,self.section1, 889 self.section3,self.section4, 890 self.section5))).hexdigest() 891 892 893 def attrs_by_section(self, sect: int, values: bool=False): 894 """ 895 Provide a tuple of attribute names for the given GRIB2 section. 896 897 Parameters 898 ---------- 899 sect 900 The GRIB2 section number. 901 values 902 Optional (default is `False`) argument to return attributes values. 903 904 Returns 905 ------- 906 attrs_by_section 907 A list of attribute names or dict of name:value pairs if `values = 908 True`. 909 """ 910 if sect in {0,1,6}: 911 attrs = templates._section_attrs[sect] 912 elif sect in {3,4,5}: 913 def _find_class_index(n): 914 _key = {3:'Grid', 4:'Product', 5:'Data'} 915 for i,c in enumerate(self.__class__.__mro__): 916 if _key[n] in c.__name__: 917 return i 918 else: 919 return [] 920 if sys.version_info.minor <= 8: 921 attrs = templates._section_attrs[sect]+\ 922 [a for a in dir(self.__class__.__mro__[_find_class_index(sect)]) if not a.startswith('_')] 923 else: 924 attrs = templates._section_attrs[sect]+\ 925 self.__class__.__mro__[_find_class_index(sect)]._attrs() 926 else: 927 attrs = [] 928 if values: 929 return {k:getattr(self,k) for k in attrs} 930 else: 931 return attrs 932 933 934 def pack(self): 935 """ 936 Pack GRIB2 section data into a binary message. 937 938 It is the user's responsibility to populate the GRIB2 section 939 information with appropriate metadata. 940 """ 941 # Create beginning of packed binary message with section 0 and 1 data. 942 self._sections = [] 943 self._msg,self._pos = g2clib.grib2_create(self.indicatorSection[2:4],self.identificationSection) 944 self._sections += [0,1] 945 946 # Add section 2 if present. 947 if isinstance(self.section2,bytes) and len(self.section2) > 0: 948 self._msg,self._pos = g2clib.grib2_addlocal(self._msg,self.section2) 949 self._sections.append(2) 950 951 # Add section 3. 952 self.section3[1] = self.nx * self.ny 953 self._msg,self._pos = g2clib.grib2_addgrid(self._msg,self.gridDefinitionSection, 954 self.gridDefinitionTemplate, 955 self._deflist) 956 self._sections.append(3) 957 958 # Prepare data. 959 if self._data is None: 960 if self._ondiskarray is None: 961 raise ValueError("Grib2Message object has no data, thus it cannot be packed.") 962 field = np.copy(self.data) 963 if self.scanModeFlags is not None: 964 if self.scanModeFlags[3]: 965 fieldsave = field.astype('f') # Casting makes a copy 966 field[1::2,:] = fieldsave[1::2,::-1] 967 fld = field.astype('f') 968 969 # Prepare bitmap, if necessary 970 bitmapflag = self.bitMapFlag.value 971 if bitmapflag == 0: 972 if self.bitmap is not None: 973 bmap = np.ravel(self.bitmap).astype(DEFAULT_NUMPY_INT) 974 else: 975 bmap = np.ravel(np.where(np.isnan(fld),0,1)).astype(DEFAULT_NUMPY_INT) 976 else: 977 bmap = None 978 979 # Prepare data for packing if nans are present 980 fld = np.ravel(fld) 981 if bitmapflag in {0,254}: 982 fld = np.where(np.isnan(fld),0,fld) 983 else: 984 if np.isnan(fld).any(): 985 if hasattr(self,'priMissingValue'): 986 fld = np.where(np.isnan(fld),self.priMissingValue,fld) 987 if hasattr(self,'_missvalmap'): 988 if hasattr(self,'priMissingValue'): 989 fld = np.where(self._missvalmap==1,self.priMissingValue,fld) 990 if hasattr(self,'secMissingValue'): 991 fld = np.where(self._missvalmap==2,self.secMissingValue,fld) 992 993 # Add sections 4, 5, 6, and 7. 994 self._msg,self._pos = g2clib.grib2_addfield(self._msg,self.pdtn, 995 self.productDefinitionTemplate, 996 self._coordlist, 997 self.drtn, 998 self.dataRepresentationTemplate, 999 fld, 1000 bitmapflag, 1001 bmap) 1002 self._sections.append(4) 1003 self._sections.append(5) 1004 self._sections.append(6) 1005 self._sections.append(7) 1006 1007 # Finalize GRIB2 message with section 8. 1008 self._msg, self._pos = g2clib.grib2_end(self._msg) 1009 self._sections.append(8) 1010 self.section0[-1] = len(self._msg) 1011 1012 1013 @property 1014 def data(self) -> np.array: 1015 """Access the unpacked data values.""" 1016 if self._data is None: 1017 if self._auto_nans != _AUTO_NANS: 1018 self._data = self._ondiskarray 1019 self._data = np.asarray(self._ondiskarray) 1020 return self._data 1021 1022 1023 @data.setter 1024 def data(self, data): 1025 if not isinstance(data, np.ndarray): 1026 raise ValueError('Grib2Message data only supports numpy arrays') 1027 self._data = data 1028 1029 1030 def flush_data(self): 1031 """ 1032 Flush the unpacked data values from the Grib2Message object. 1033 1034 Note: If the Grib2Message object was constructed from "scratch" (i.e. 1035 not read from file), this method will remove the data array from 1036 the object and it cannot be recovered. 1037 """ 1038 self._data = None 1039 self.bitmap = None 1040 1041 1042 def __getitem__(self, item): 1043 return self.data[item] 1044 1045 1046 def __setitem__(self, item): 1047 raise NotImplementedError('assignment of data not supported via setitem') 1048 1049 1050 def latlons(self, *args, **kwrgs): 1051 """Alias for `grib2io.Grib2Message.grid` method.""" 1052 return self.grid(*args, **kwrgs) 1053 1054 1055 def grid(self, unrotate: bool=True): 1056 """ 1057 Return lats,lons (in degrees) of grid. 1058 1059 Currently can handle reg. lat/lon,cglobal Gaussian, mercator, 1060 stereographic, lambert conformal, albers equal-area, space-view and 1061 azimuthal equidistant grids. 1062 1063 Parameters 1064 ---------- 1065 unrotate 1066 If `True` [DEFAULT], and grid is rotated lat/lon, then unrotate the 1067 grid, otherwise `False`, do not. 1068 1069 Returns 1070 ------- 1071 lats, lons : numpy.ndarray 1072 Returns two numpy.ndarrays with dtype=numpy.float32 of grid 1073 latitudes and longitudes in units of degrees. 1074 """ 1075 if self._sha1_section3 in _latlon_datastore.keys(): 1076 return (_latlon_datastore[self._sha1_section3]['latitude'], 1077 _latlon_datastore[self._sha1_section3]['longitude']) 1078 gdtn = self.gridDefinitionTemplateNumber.value 1079 gdtmpl = self.gridDefinitionTemplate 1080 reggrid = self.gridDefinitionSection[2] == 0 # This means regular 2-d grid 1081 if gdtn == 0: 1082 # Regular lat/lon grid 1083 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1084 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1085 dlon = self.gridlengthXDirection 1086 dlat = self.gridlengthYDirection 1087 if lon2 < lon1 and dlon < 0: lon1 = -lon1 1088 lats = np.linspace(lat1,lat2,self.ny) 1089 if reggrid: 1090 lons = np.linspace(lon1,lon2,self.nx) 1091 else: 1092 lons = np.linspace(lon1,lon2,self.ny*2) 1093 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1094 elif gdtn == 1: # Rotated Lat/Lon grid 1095 pj = pyproj.Proj(self.projParameters) 1096 lat1,lon1 = self.latitudeFirstGridpoint,self.longitudeFirstGridpoint 1097 lat2,lon2 = self.latitudeLastGridpoint,self.longitudeLastGridpoint 1098 if lon1 > 180.0: lon1 -= 360.0 1099 if lon2 > 180.0: lon2 -= 360.0 1100 lats = np.linspace(lat1,lat2,self.ny) 1101 lons = np.linspace(lon1,lon2,self.nx) 1102 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1103 if unrotate: 1104 from grib2io.utils import rotated_grid 1105 lats,lons = rotated_grid.unrotate(lats,lons,self.anglePoleRotation, 1106 self.latitudeSouthernPole, 1107 self.longitudeSouthernPole) 1108 elif gdtn == 40: # Gaussian grid (only works for global!) 1109 from grib2io.utils.gauss_grid import gaussian_latitudes 1110 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1111 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1112 nlats = self.ny 1113 if not reggrid: # Reduced Gaussian grid. 1114 nlons = 2*nlats 1115 dlon = 360./nlons 1116 else: 1117 nlons = self.nx 1118 dlon = self.gridlengthXDirection 1119 lons = np.linspace(lon1,lon2,nlons) 1120 # Compute Gaussian lats (north to south) 1121 lats = gaussian_latitudes(nlats) 1122 if lat1 < lat2: # reverse them if necessary 1123 lats = lats[::-1] 1124 lons,lats = np.meshgrid(lons,lats) 1125 elif gdtn in {10,20,30,31,110}: 1126 # Mercator, Lambert Conformal, Stereographic, Albers Equal Area, 1127 # Azimuthal Equidistant 1128 dx,dy = self.gridlengthXDirection, self.gridlengthYDirection 1129 lon1,lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1130 pj = pyproj.Proj(self.projParameters) 1131 llcrnrx, llcrnry = pj(lon1,lat1) 1132 x = llcrnrx+dx*np.arange(self.nx) 1133 y = llcrnry+dy*np.arange(self.ny) 1134 x,y = np.meshgrid(x, y) 1135 lons,lats = pj(x, y, inverse=True) 1136 elif gdtn == 90: 1137 # Satellite Projection 1138 dx = self.gridlengthXDirection 1139 dy = self.gridlengthYDirection 1140 pj = pyproj.Proj(self.projParameters) 1141 x = dx*np.indices((self.ny,self.nx),'f')[1,:,:] 1142 x -= 0.5*x.max() 1143 y = dy*np.indices((self.ny,self.nx),'f')[0,:,:] 1144 y -= 0.5*y.max() 1145 lons,lats = pj(x,y,inverse=True) 1146 # Set lons,lats to 1.e30 where undefined 1147 abslons = np.fabs(lons) 1148 abslats = np.fabs(lats) 1149 lons = np.where(abslons < 1.e20, lons, 1.e30) 1150 lats = np.where(abslats < 1.e20, lats, 1.e30) 1151 elif gdtn == 32769: 1152 # Special NCEP Grid, Rotated Lat/Lon, Arakawa E-Grid (Non-Staggered) 1153 from grib2io.utils import arakawa_rotated_grid 1154 from grib2io.utils.rotated_grid import DEG2RAD 1155 di, dj = 0.0, 0.0 1156 do_180 = False 1157 idir = 1 if self.scanModeFlags[0] == 0 else -1 1158 jdir = -1 if self.scanModeFlags[1] == 0 else 1 1159 grid_oriented = 0 if self.resolutionAndComponentFlags[4] == 0 else 1 1160 do_rot = 1 if grid_oriented == 1 else 0 1161 la1 = self.latitudeFirstGridpoint 1162 lo1 = self.longitudeFirstGridpoint 1163 clon = self.longitudeCenterGridpoint 1164 clat = self.latitudeCenterGridpoint 1165 lasp = clat - 90.0 1166 losp = clon 1167 llat, llon = arakawa_rotated_grid.ll2rot(la1,lo1,lasp,losp) 1168 la2, lo2 = arakawa_rotated_grid.rot2ll(-llat,-llon,lasp,losp) 1169 rlat = -llat 1170 rlon = -llon 1171 if self.nx == 1: 1172 di = 0.0 1173 elif idir == 1: 1174 ti = rlon 1175 while ti < llon: 1176 ti += 360.0 1177 di = (ti - llon)/float(self.nx-1) 1178 else: 1179 ti = llon 1180 while ti < rlon: 1181 ti += 360.0 1182 di = (ti - rlon)/float(self.nx-1) 1183 if self.ny == 1: 1184 dj = 0.0 1185 else: 1186 dj = (rlat - llat)/float(self.ny-1) 1187 if dj < 0.0: 1188 dj = -dj 1189 if idir == 1: 1190 if llon > rlon: 1191 llon -= 360.0 1192 if llon < 0 and rlon > 0: 1193 do_180 = True 1194 else: 1195 if rlon > llon: 1196 rlon -= 360.0 1197 if rlon < 0 and llon > 0: 1198 do_180 = True 1199 xlat1d = llat + (np.arange(self.ny)*jdir*dj) 1200 xlon1d = llon + (np.arange(self.nx)*idir*di) 1201 xlons, xlats = np.meshgrid(xlon1d,xlat1d) 1202 rot2ll_vectorized = np.vectorize(arakawa_rotated_grid.rot2ll) 1203 lats, lons = rot2ll_vectorized(xlats,xlons,lasp,losp) 1204 if do_180: 1205 lons = np.where(lons>180.0,lons-360.0,lons) 1206 vector_rotation_angles_vectorized = np.vectorize(arakawa_rotated_grid.vector_rotation_angles) 1207 rots = vector_rotation_angles_vectorized(lats, lons, clat, losp, xlats) 1208 del xlat1d, xlon1d, xlats, xlons 1209 else: 1210 raise ValueError('Unsupported grid') 1211 1212 _latlon_datastore[self._sha1_section3] = dict(latitude=lats,longitude=lons) 1213 try: 1214 _latlon_datastore[self._sha1_section3]['vector_rotation_angles'] = rots 1215 except(NameError): 1216 pass 1217 1218 return lats, lons 1219 1220 1221 def map_keys(self): 1222 """ 1223 Unpack data grid replacing integer values with strings. 1224 1225 These types of fields are categorical or classifications where data 1226 values do not represent an observable or predictable physical quantity. 1227 An example of such a field would be [Dominant Precipitation Type - 1228 DPTYPE](https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table4-201.shtml) 1229 1230 Returns 1231 ------- 1232 map_keys 1233 numpy.ndarray of string values per element. 1234 """ 1235 hold_auto_nans = _AUTO_NANS 1236 set_auto_nans(False) 1237 if (np.all(self.section1[0:2]==[7,14]) and self.shortName == 'PWTHER') or \ 1238 (np.all(self.section1[0:2]==[8,65535]) and self.shortName == 'WX'): 1239 keys = utils.decode_wx_strings(self.section2) 1240 if hasattr(self,'priMissingValue') and self.priMissingValue not in [None,0]: 1241 keys[int(self.priMissingValue)] = 'Missing' 1242 if hasattr(self,'secMissingValue') and self.secMissingValue not in [None,0]: 1243 keys[int(self.secMissingValue)] = 'Missing' 1244 u,inv = np.unique(self.data,return_inverse=True) 1245 fld = np.array([keys[x] for x in u])[inv].reshape(self.data.shape) 1246 else: 1247 # For data whose units are defined in a code table (i.e. classification or mask) 1248 tblname = re.findall(r'\d\.\d+',self.units,re.IGNORECASE)[0] 1249 fld = self.data.astype(np.int32).astype(str) 1250 tbl = tables.get_table(tblname,expand=True) 1251 for val in np.unique(fld): 1252 fld = np.where(fld==val,tbl[val],fld) 1253 set_auto_nans(hold_auto_nans) 1254 return fld 1255 1256 1257 def to_bytes(self, validate: bool=True): 1258 """ 1259 Return packed GRIB2 message in bytes format. 1260 1261 This will be useful for exporting data in non-file formats. For example, 1262 can be used to output grib data directly to S3 using the boto3 client 1263 without the need to write a temporary file to upload first. 1264 1265 Parameters 1266 ---------- 1267 validate: default=True 1268 If `True` (DEFAULT), validates first/last four bytes for proper 1269 formatting, else returns None. If `False`, message is output as is. 1270 1271 Returns 1272 ------- 1273 to_bytes 1274 Returns GRIB2 formatted message as bytes. 1275 """ 1276 if hasattr(self,'_msg'): 1277 if validate: 1278 if self.validate(): 1279 return self._msg 1280 else: 1281 return None 1282 else: 1283 return self._msg 1284 else: 1285 return None 1286 1287 1288 def interpolate(self, method, grid_def_out, method_options=None, drtn=None, 1289 num_threads=1): 1290 """ 1291 Grib2Message Interpolator 1292 1293 Performs spatial interpolation via the [NCEPLIBS-ip 1294 library](https://github.com/NOAA-EMC/NCEPLIBS-ip). This interpolate 1295 method only supports scalar interpolation. If you need to perform 1296 vector interpolation, use the module-level `grib2io.interpolate` 1297 function. 1298 1299 Parameters 1300 ---------- 1301 method 1302 Interpolate method to use. This can either be an integer or string 1303 using the following mapping: 1304 1305 | Interpolate Scheme | Integer Value | 1306 | :---: | :---: | 1307 | 'bilinear' | 0 | 1308 | 'bicubic' | 1 | 1309 | 'neighbor' | 2 | 1310 | 'budget' | 3 | 1311 | 'spectral' | 4 | 1312 | 'neighbor-budget' | 6 | 1313 1314 grid_def_out : grib2io.Grib2GridDef 1315 Grib2GridDef object of the output grid. 1316 method_options : list of ints, optional 1317 Interpolation options. See the NCEPLIBS-ip documentation for 1318 more information on how these are used. 1319 drtn 1320 Data Representation Template to be used for the returned 1321 interpolated GRIB2 message. When `None`, the data representation 1322 template of the source GRIB2 message is used. Once again, it is the 1323 user's responsibility to properly set the Data Representation 1324 Template attributes. 1325 num_threads : int, optional 1326 Number of OpenMP threads to use for interpolation. The default 1327 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1328 was not built with OpenMP, then this keyword argument and value 1329 will have no impact. 1330 1331 Returns 1332 ------- 1333 interpolate 1334 If interpolating to a grid, a new Grib2Message object is returned. 1335 The GRIB2 metadata of the new Grib2Message object is identical to 1336 the input except where required to be different because of the new 1337 grid specs and possibly a new data representation template. 1338 1339 If interpolating to station points, the interpolated data values are 1340 returned as a numpy.ndarray. 1341 """ 1342 section0 = self.section0 1343 section0[-1] = 0 1344 gds = [0, grid_def_out.npoints, 0, 255, grid_def_out.gdtn] 1345 section3 = np.concatenate((gds,grid_def_out.gdt)) 1346 drtn = self.drtn if drtn is None else drtn 1347 1348 msg = Grib2Message(section0,self.section1,self.section2,section3, 1349 self.section4,None,self.bitMapFlag.value,drtn=drtn) 1350 1351 msg._msgnum = -1 1352 msg._deflist = self._deflist 1353 msg._coordlist = self._coordlist 1354 shape = (msg.ny,msg.nx) 1355 ndim = 2 1356 if msg.typeOfValues == 0: 1357 dtype = 'float32' 1358 elif msg.typeOfValues == 1: 1359 dtype = 'int32' 1360 msg._data = interpolate(self.data,method,Grib2GridDef.from_section3(self.section3),grid_def_out, 1361 method_options=method_options,num_threads=num_threads).reshape(msg.ny,msg.nx) 1362 msg.section5[0] = grid_def_out.npoints 1363 return msg 1364 1365 1366 def subset(self, lats, lons): 1367 """ 1368 Return a spatial subset. 1369 1370 Currently only supports regular grids of the following types: 1371 1372 | Grid Type | gdtn | 1373 | :---: | :---: | 1374 | Latitude/Longitude, Equidistant Cylindrical, or Plate Carree | 0 | 1375 | Rotated Latitude/Longitude | 1 | 1376 | Mercator | 10 | 1377 | Polar Stereographic | 20 | 1378 | Lambert Conformal | 30 | 1379 | Albers Equal-Area | 31 | 1380 | Gaussian Latitude/Longitude | 40 | 1381 | Equatorial Azimuthal Equidistant Projection | 110 | 1382 1383 Parameters 1384 ---------- 1385 lats 1386 List or tuple of latitudes. The minimum and maximum latitudes will 1387 be used to define the southern and northern boundaries. 1388 1389 The order of the latitudes is not important. The function will 1390 determine which is the minimum and maximum. 1391 1392 The latitudes should be in decimal degrees with 0.0 at the equator, 1393 positive values in the northern hemisphere increasing to 90, and 1394 negative values in the southern hemisphere decreasing to -90. 1395 lons 1396 List or tuple of longitudes. The minimum and maximum longitudes 1397 will be used to define the western and eastern boundaries. 1398 1399 The order of the longitudes is not important. The function will 1400 determine which is the minimum and maximum. 1401 1402 GRIB2 longitudes should be in decimal degrees with 0.0 at the prime 1403 meridian, positive values increasing eastward to 360. There are no 1404 negative GRIB2 longitudes. 1405 1406 The typical west longitudes that start at 0.0 at the prime meridian 1407 and decrease to -180 westward, are converted to GRIB2 longitudes by 1408 '360 - (absolute value of the west longitude)' where typical 1409 eastern longitudes are unchanged as GRIB2 longitudes. 1410 1411 Returns 1412 ------- 1413 subset 1414 A spatial subset of a GRIB2 message. 1415 """ 1416 if self.gdtn not in [0, 1, 10, 20, 30, 31, 40, 110]: 1417 raise ValueError( 1418 """ 1419 1420Subset only works for 1421 Latitude/Longitude, Equidistant Cylindrical, or Plate Carree (gdtn=0) 1422 Rotated Latitude/Longitude (gdtn=1) 1423 Mercator (gdtn=10) 1424 Polar Stereographic (gdtn=20) 1425 Lambert Conformal (gdtn=30) 1426 Albers Equal-Area (gdtn=31) 1427 Gaussian Latitude/Longitude (gdtn=40) 1428 Equatorial Azimuthal Equidistant Projection (gdtn=110) 1429 1430""" 1431 ) 1432 1433 if self.nx == 0 or self.ny == 0: 1434 raise ValueError( 1435 """ 1436 1437Subset only works for regular grids. 1438 1439""" 1440 ) 1441 1442 newmsg = Grib2Message( 1443 np.copy(self.section0), 1444 np.copy(self.section1), 1445 np.copy(self.section2), 1446 np.copy(self.section3), 1447 np.copy(self.section4), 1448 np.copy(self.section5), 1449 ) 1450 1451 msglats, msglons = self.grid() 1452 1453 la1 = np.max(lats) 1454 lo1 = np.min(lons) 1455 la2 = np.min(lats) 1456 lo2 = np.max(lons) 1457 1458 # Find the indices of the first and last grid points to the nearest 1459 # lat/lon values in the grid. 1460 first_lat = np.abs(msglats - la1) 1461 first_lon = np.abs(msglons - lo1) 1462 max_idx = np.maximum(first_lat, first_lon) 1463 first_j, first_i = np.where(max_idx == np.min(max_idx)) 1464 1465 last_lat = np.abs(msglats - la2) 1466 last_lon = np.abs(msglons - lo2) 1467 max_idx = np.maximum(last_lat, last_lon) 1468 last_j, last_i = np.where(max_idx == np.min(max_idx)) 1469 1470 setattr(newmsg, "latitudeFirstGridpoint", msglats[first_j[0], first_i[0]]) 1471 setattr(newmsg, "longitudeFirstGridpoint", msglons[first_j[0], first_i[0]]) 1472 setattr(newmsg, "nx", np.abs(first_i[0] - last_i[0])) 1473 setattr(newmsg, "ny", np.abs(first_j[0] - last_j[0])) 1474 1475 # Set *LastGridpoint attributes even if only used for gdtn=[0, 1, 40]. 1476 # This information is used to subset xarray datasets and even though 1477 # unnecessary for some supported grid types, it won't affect a grib2io 1478 # message to set them. 1479 setattr(newmsg, "latitudeLastGridpoint", msglats[last_j[0], last_i[0]]) 1480 setattr(newmsg, "longitudeLastGridpoint", msglons[last_j[0], last_i[0]]) 1481 1482 setattr( 1483 newmsg, 1484 "data", 1485 self.data[ 1486 min(first_j[0], last_j[0]) : max(first_j[0], last_j[0]), 1487 min(first_i[0], last_i[0]) : max(first_i[0], last_i[0]), 1488 ].copy(), 1489 ) 1490 1491 # Need to set the newmsg._sha1_section3 to a blank string so the grid 1492 # method ignores the cached lat/lon values. This will force the grid 1493 # method to recompute the lat/lon values for the subsetted grid. 1494 newmsg._sha1_section3 = "" 1495 newmsg.grid() 1496 1497 return newmsg 1498 1499 def validate(self): 1500 """ 1501 Validate a complete GRIB2 message. 1502 1503 The g2c library does its own internal validation when g2_gribend() is called, but 1504 we will check in grib2io also. The validation checks if the first 4 bytes in 1505 self._msg is 'GRIB' and '7777' as the last 4 bytes and that the message length in 1506 section 0 equals the length of the packed message. 1507 1508 Returns 1509 ------- 1510 `True` if the packed GRIB2 message is complete and well-formed, `False` otherwise. 1511 """ 1512 valid = False 1513 if hasattr(self,'_msg'): 1514 if self._msg[0:4]+self._msg[-4:] == b'GRIB7777': 1515 if self.section0[-1] == len(self._msg): 1516 valid = True 1517 return valid
GRIB2 Message base class.
GRIB2 Section 1, Identification Section
GRIB2 Section 3, Grid Definition Section
Product Definition Template
766 @property 767 def gdtn(self): 768 """Return Grid Definition Template Number""" 769 return self.section3[4]
Return Grid Definition Template Number
772 @property 773 def gdt(self): 774 """Return Grid Definition Template.""" 775 return self.gridDefinitionTemplate
Return Grid Definition Template.
778 @property 779 def pdtn(self): 780 """Return Product Definition Template Number.""" 781 return self.section4[1]
Return Product Definition Template Number.
784 @property 785 def pdt(self): 786 """Return Product Definition Template.""" 787 return self.productDefinitionTemplate
Return Product Definition Template.
790 @property 791 def drtn(self): 792 """Return Data Representation Template Number.""" 793 return self.section5[1]
Return Data Representation Template Number.
796 @property 797 def drt(self): 798 """Return Data Representation Template.""" 799 return self.dataRepresentationTemplate
Return Data Representation Template.
802 @property 803 def pdy(self): 804 """Return the PDY ('YYYYMMDD').""" 805 return ''.join([str(i) for i in self.section1[5:8]])
Return the PDY ('YYYYMMDD').
808 @property 809 def griddef(self): 810 """Return a Grib2GridDef instance for a GRIB2 message.""" 811 return Grib2GridDef.from_section3(self.section3)
Return a Grib2GridDef instance for a GRIB2 message.
825 @property 826 def min(self): 827 """Return minimum value of data.""" 828 return np.nanmin(self.data)
Return minimum value of data.
831 @property 832 def max(self): 833 """Return maximum value of data.""" 834 return np.nanmax(self.data)
Return maximum value of data.
837 @property 838 def mean(self): 839 """Return mean value of data.""" 840 return np.nanmean(self.data)
Return mean value of data.
843 @property 844 def median(self): 845 """Return median value of data.""" 846 return np.nanmedian(self.data)
Return median value of data.
893 def attrs_by_section(self, sect: int, values: bool=False): 894 """ 895 Provide a tuple of attribute names for the given GRIB2 section. 896 897 Parameters 898 ---------- 899 sect 900 The GRIB2 section number. 901 values 902 Optional (default is `False`) argument to return attributes values. 903 904 Returns 905 ------- 906 attrs_by_section 907 A list of attribute names or dict of name:value pairs if `values = 908 True`. 909 """ 910 if sect in {0,1,6}: 911 attrs = templates._section_attrs[sect] 912 elif sect in {3,4,5}: 913 def _find_class_index(n): 914 _key = {3:'Grid', 4:'Product', 5:'Data'} 915 for i,c in enumerate(self.__class__.__mro__): 916 if _key[n] in c.__name__: 917 return i 918 else: 919 return [] 920 if sys.version_info.minor <= 8: 921 attrs = templates._section_attrs[sect]+\ 922 [a for a in dir(self.__class__.__mro__[_find_class_index(sect)]) if not a.startswith('_')] 923 else: 924 attrs = templates._section_attrs[sect]+\ 925 self.__class__.__mro__[_find_class_index(sect)]._attrs() 926 else: 927 attrs = [] 928 if values: 929 return {k:getattr(self,k) for k in attrs} 930 else: 931 return attrs
Provide a tuple of attribute names for the given GRIB2 section.
Parameters
- sect: The GRIB2 section number.
- values: Optional (default is
False
) argument to return attributes values.
Returns
- attrs_by_section: A list of attribute names or dict of name:value pairs if
values = True
.
934 def pack(self): 935 """ 936 Pack GRIB2 section data into a binary message. 937 938 It is the user's responsibility to populate the GRIB2 section 939 information with appropriate metadata. 940 """ 941 # Create beginning of packed binary message with section 0 and 1 data. 942 self._sections = [] 943 self._msg,self._pos = g2clib.grib2_create(self.indicatorSection[2:4],self.identificationSection) 944 self._sections += [0,1] 945 946 # Add section 2 if present. 947 if isinstance(self.section2,bytes) and len(self.section2) > 0: 948 self._msg,self._pos = g2clib.grib2_addlocal(self._msg,self.section2) 949 self._sections.append(2) 950 951 # Add section 3. 952 self.section3[1] = self.nx * self.ny 953 self._msg,self._pos = g2clib.grib2_addgrid(self._msg,self.gridDefinitionSection, 954 self.gridDefinitionTemplate, 955 self._deflist) 956 self._sections.append(3) 957 958 # Prepare data. 959 if self._data is None: 960 if self._ondiskarray is None: 961 raise ValueError("Grib2Message object has no data, thus it cannot be packed.") 962 field = np.copy(self.data) 963 if self.scanModeFlags is not None: 964 if self.scanModeFlags[3]: 965 fieldsave = field.astype('f') # Casting makes a copy 966 field[1::2,:] = fieldsave[1::2,::-1] 967 fld = field.astype('f') 968 969 # Prepare bitmap, if necessary 970 bitmapflag = self.bitMapFlag.value 971 if bitmapflag == 0: 972 if self.bitmap is not None: 973 bmap = np.ravel(self.bitmap).astype(DEFAULT_NUMPY_INT) 974 else: 975 bmap = np.ravel(np.where(np.isnan(fld),0,1)).astype(DEFAULT_NUMPY_INT) 976 else: 977 bmap = None 978 979 # Prepare data for packing if nans are present 980 fld = np.ravel(fld) 981 if bitmapflag in {0,254}: 982 fld = np.where(np.isnan(fld),0,fld) 983 else: 984 if np.isnan(fld).any(): 985 if hasattr(self,'priMissingValue'): 986 fld = np.where(np.isnan(fld),self.priMissingValue,fld) 987 if hasattr(self,'_missvalmap'): 988 if hasattr(self,'priMissingValue'): 989 fld = np.where(self._missvalmap==1,self.priMissingValue,fld) 990 if hasattr(self,'secMissingValue'): 991 fld = np.where(self._missvalmap==2,self.secMissingValue,fld) 992 993 # Add sections 4, 5, 6, and 7. 994 self._msg,self._pos = g2clib.grib2_addfield(self._msg,self.pdtn, 995 self.productDefinitionTemplate, 996 self._coordlist, 997 self.drtn, 998 self.dataRepresentationTemplate, 999 fld, 1000 bitmapflag, 1001 bmap) 1002 self._sections.append(4) 1003 self._sections.append(5) 1004 self._sections.append(6) 1005 self._sections.append(7) 1006 1007 # Finalize GRIB2 message with section 8. 1008 self._msg, self._pos = g2clib.grib2_end(self._msg) 1009 self._sections.append(8) 1010 self.section0[-1] = len(self._msg)
Pack GRIB2 section data into a binary message.
It is the user's responsibility to populate the GRIB2 section information with appropriate metadata.
1013 @property 1014 def data(self) -> np.array: 1015 """Access the unpacked data values.""" 1016 if self._data is None: 1017 if self._auto_nans != _AUTO_NANS: 1018 self._data = self._ondiskarray 1019 self._data = np.asarray(self._ondiskarray) 1020 return self._data
Access the unpacked data values.
1030 def flush_data(self): 1031 """ 1032 Flush the unpacked data values from the Grib2Message object. 1033 1034 Note: If the Grib2Message object was constructed from "scratch" (i.e. 1035 not read from file), this method will remove the data array from 1036 the object and it cannot be recovered. 1037 """ 1038 self._data = None 1039 self.bitmap = None
Flush the unpacked data values from the Grib2Message object.
Note: If the Grib2Message object was constructed from "scratch" (i.e. not read from file), this method will remove the data array from the object and it cannot be recovered.
1050 def latlons(self, *args, **kwrgs): 1051 """Alias for `grib2io.Grib2Message.grid` method.""" 1052 return self.grid(*args, **kwrgs)
Alias for grib2io.Grib2Message.grid
method.
1055 def grid(self, unrotate: bool=True): 1056 """ 1057 Return lats,lons (in degrees) of grid. 1058 1059 Currently can handle reg. lat/lon,cglobal Gaussian, mercator, 1060 stereographic, lambert conformal, albers equal-area, space-view and 1061 azimuthal equidistant grids. 1062 1063 Parameters 1064 ---------- 1065 unrotate 1066 If `True` [DEFAULT], and grid is rotated lat/lon, then unrotate the 1067 grid, otherwise `False`, do not. 1068 1069 Returns 1070 ------- 1071 lats, lons : numpy.ndarray 1072 Returns two numpy.ndarrays with dtype=numpy.float32 of grid 1073 latitudes and longitudes in units of degrees. 1074 """ 1075 if self._sha1_section3 in _latlon_datastore.keys(): 1076 return (_latlon_datastore[self._sha1_section3]['latitude'], 1077 _latlon_datastore[self._sha1_section3]['longitude']) 1078 gdtn = self.gridDefinitionTemplateNumber.value 1079 gdtmpl = self.gridDefinitionTemplate 1080 reggrid = self.gridDefinitionSection[2] == 0 # This means regular 2-d grid 1081 if gdtn == 0: 1082 # Regular lat/lon grid 1083 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1084 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1085 dlon = self.gridlengthXDirection 1086 dlat = self.gridlengthYDirection 1087 if lon2 < lon1 and dlon < 0: lon1 = -lon1 1088 lats = np.linspace(lat1,lat2,self.ny) 1089 if reggrid: 1090 lons = np.linspace(lon1,lon2,self.nx) 1091 else: 1092 lons = np.linspace(lon1,lon2,self.ny*2) 1093 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1094 elif gdtn == 1: # Rotated Lat/Lon grid 1095 pj = pyproj.Proj(self.projParameters) 1096 lat1,lon1 = self.latitudeFirstGridpoint,self.longitudeFirstGridpoint 1097 lat2,lon2 = self.latitudeLastGridpoint,self.longitudeLastGridpoint 1098 if lon1 > 180.0: lon1 -= 360.0 1099 if lon2 > 180.0: lon2 -= 360.0 1100 lats = np.linspace(lat1,lat2,self.ny) 1101 lons = np.linspace(lon1,lon2,self.nx) 1102 lons,lats = np.meshgrid(lons,lats) # Make 2-d arrays. 1103 if unrotate: 1104 from grib2io.utils import rotated_grid 1105 lats,lons = rotated_grid.unrotate(lats,lons,self.anglePoleRotation, 1106 self.latitudeSouthernPole, 1107 self.longitudeSouthernPole) 1108 elif gdtn == 40: # Gaussian grid (only works for global!) 1109 from grib2io.utils.gauss_grid import gaussian_latitudes 1110 lon1, lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1111 lon2, lat2 = self.longitudeLastGridpoint, self.latitudeLastGridpoint 1112 nlats = self.ny 1113 if not reggrid: # Reduced Gaussian grid. 1114 nlons = 2*nlats 1115 dlon = 360./nlons 1116 else: 1117 nlons = self.nx 1118 dlon = self.gridlengthXDirection 1119 lons = np.linspace(lon1,lon2,nlons) 1120 # Compute Gaussian lats (north to south) 1121 lats = gaussian_latitudes(nlats) 1122 if lat1 < lat2: # reverse them if necessary 1123 lats = lats[::-1] 1124 lons,lats = np.meshgrid(lons,lats) 1125 elif gdtn in {10,20,30,31,110}: 1126 # Mercator, Lambert Conformal, Stereographic, Albers Equal Area, 1127 # Azimuthal Equidistant 1128 dx,dy = self.gridlengthXDirection, self.gridlengthYDirection 1129 lon1,lat1 = self.longitudeFirstGridpoint, self.latitudeFirstGridpoint 1130 pj = pyproj.Proj(self.projParameters) 1131 llcrnrx, llcrnry = pj(lon1,lat1) 1132 x = llcrnrx+dx*np.arange(self.nx) 1133 y = llcrnry+dy*np.arange(self.ny) 1134 x,y = np.meshgrid(x, y) 1135 lons,lats = pj(x, y, inverse=True) 1136 elif gdtn == 90: 1137 # Satellite Projection 1138 dx = self.gridlengthXDirection 1139 dy = self.gridlengthYDirection 1140 pj = pyproj.Proj(self.projParameters) 1141 x = dx*np.indices((self.ny,self.nx),'f')[1,:,:] 1142 x -= 0.5*x.max() 1143 y = dy*np.indices((self.ny,self.nx),'f')[0,:,:] 1144 y -= 0.5*y.max() 1145 lons,lats = pj(x,y,inverse=True) 1146 # Set lons,lats to 1.e30 where undefined 1147 abslons = np.fabs(lons) 1148 abslats = np.fabs(lats) 1149 lons = np.where(abslons < 1.e20, lons, 1.e30) 1150 lats = np.where(abslats < 1.e20, lats, 1.e30) 1151 elif gdtn == 32769: 1152 # Special NCEP Grid, Rotated Lat/Lon, Arakawa E-Grid (Non-Staggered) 1153 from grib2io.utils import arakawa_rotated_grid 1154 from grib2io.utils.rotated_grid import DEG2RAD 1155 di, dj = 0.0, 0.0 1156 do_180 = False 1157 idir = 1 if self.scanModeFlags[0] == 0 else -1 1158 jdir = -1 if self.scanModeFlags[1] == 0 else 1 1159 grid_oriented = 0 if self.resolutionAndComponentFlags[4] == 0 else 1 1160 do_rot = 1 if grid_oriented == 1 else 0 1161 la1 = self.latitudeFirstGridpoint 1162 lo1 = self.longitudeFirstGridpoint 1163 clon = self.longitudeCenterGridpoint 1164 clat = self.latitudeCenterGridpoint 1165 lasp = clat - 90.0 1166 losp = clon 1167 llat, llon = arakawa_rotated_grid.ll2rot(la1,lo1,lasp,losp) 1168 la2, lo2 = arakawa_rotated_grid.rot2ll(-llat,-llon,lasp,losp) 1169 rlat = -llat 1170 rlon = -llon 1171 if self.nx == 1: 1172 di = 0.0 1173 elif idir == 1: 1174 ti = rlon 1175 while ti < llon: 1176 ti += 360.0 1177 di = (ti - llon)/float(self.nx-1) 1178 else: 1179 ti = llon 1180 while ti < rlon: 1181 ti += 360.0 1182 di = (ti - rlon)/float(self.nx-1) 1183 if self.ny == 1: 1184 dj = 0.0 1185 else: 1186 dj = (rlat - llat)/float(self.ny-1) 1187 if dj < 0.0: 1188 dj = -dj 1189 if idir == 1: 1190 if llon > rlon: 1191 llon -= 360.0 1192 if llon < 0 and rlon > 0: 1193 do_180 = True 1194 else: 1195 if rlon > llon: 1196 rlon -= 360.0 1197 if rlon < 0 and llon > 0: 1198 do_180 = True 1199 xlat1d = llat + (np.arange(self.ny)*jdir*dj) 1200 xlon1d = llon + (np.arange(self.nx)*idir*di) 1201 xlons, xlats = np.meshgrid(xlon1d,xlat1d) 1202 rot2ll_vectorized = np.vectorize(arakawa_rotated_grid.rot2ll) 1203 lats, lons = rot2ll_vectorized(xlats,xlons,lasp,losp) 1204 if do_180: 1205 lons = np.where(lons>180.0,lons-360.0,lons) 1206 vector_rotation_angles_vectorized = np.vectorize(arakawa_rotated_grid.vector_rotation_angles) 1207 rots = vector_rotation_angles_vectorized(lats, lons, clat, losp, xlats) 1208 del xlat1d, xlon1d, xlats, xlons 1209 else: 1210 raise ValueError('Unsupported grid') 1211 1212 _latlon_datastore[self._sha1_section3] = dict(latitude=lats,longitude=lons) 1213 try: 1214 _latlon_datastore[self._sha1_section3]['vector_rotation_angles'] = rots 1215 except(NameError): 1216 pass 1217 1218 return lats, lons
Return lats,lons (in degrees) of grid.
Currently can handle reg. lat/lon,cglobal Gaussian, mercator, stereographic, lambert conformal, albers equal-area, space-view and azimuthal equidistant grids.
Parameters
- unrotate: If
True
[DEFAULT], and grid is rotated lat/lon, then unrotate the grid, otherwiseFalse
, do not.
Returns
- lats, lons (numpy.ndarray): Returns two numpy.ndarrays with dtype=numpy.float32 of grid latitudes and longitudes in units of degrees.
1221 def map_keys(self): 1222 """ 1223 Unpack data grid replacing integer values with strings. 1224 1225 These types of fields are categorical or classifications where data 1226 values do not represent an observable or predictable physical quantity. 1227 An example of such a field would be [Dominant Precipitation Type - 1228 DPTYPE](https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_table4-201.shtml) 1229 1230 Returns 1231 ------- 1232 map_keys 1233 numpy.ndarray of string values per element. 1234 """ 1235 hold_auto_nans = _AUTO_NANS 1236 set_auto_nans(False) 1237 if (np.all(self.section1[0:2]==[7,14]) and self.shortName == 'PWTHER') or \ 1238 (np.all(self.section1[0:2]==[8,65535]) and self.shortName == 'WX'): 1239 keys = utils.decode_wx_strings(self.section2) 1240 if hasattr(self,'priMissingValue') and self.priMissingValue not in [None,0]: 1241 keys[int(self.priMissingValue)] = 'Missing' 1242 if hasattr(self,'secMissingValue') and self.secMissingValue not in [None,0]: 1243 keys[int(self.secMissingValue)] = 'Missing' 1244 u,inv = np.unique(self.data,return_inverse=True) 1245 fld = np.array([keys[x] for x in u])[inv].reshape(self.data.shape) 1246 else: 1247 # For data whose units are defined in a code table (i.e. classification or mask) 1248 tblname = re.findall(r'\d\.\d+',self.units,re.IGNORECASE)[0] 1249 fld = self.data.astype(np.int32).astype(str) 1250 tbl = tables.get_table(tblname,expand=True) 1251 for val in np.unique(fld): 1252 fld = np.where(fld==val,tbl[val],fld) 1253 set_auto_nans(hold_auto_nans) 1254 return fld
Unpack data grid replacing integer values with strings.
These types of fields are categorical or classifications where data values do not represent an observable or predictable physical quantity. An example of such a field would be Dominant Precipitation Type - DPTYPE
Returns
- map_keys: numpy.ndarray of string values per element.
1257 def to_bytes(self, validate: bool=True): 1258 """ 1259 Return packed GRIB2 message in bytes format. 1260 1261 This will be useful for exporting data in non-file formats. For example, 1262 can be used to output grib data directly to S3 using the boto3 client 1263 without the need to write a temporary file to upload first. 1264 1265 Parameters 1266 ---------- 1267 validate: default=True 1268 If `True` (DEFAULT), validates first/last four bytes for proper 1269 formatting, else returns None. If `False`, message is output as is. 1270 1271 Returns 1272 ------- 1273 to_bytes 1274 Returns GRIB2 formatted message as bytes. 1275 """ 1276 if hasattr(self,'_msg'): 1277 if validate: 1278 if self.validate(): 1279 return self._msg 1280 else: 1281 return None 1282 else: 1283 return self._msg 1284 else: 1285 return None
Return packed GRIB2 message in bytes format.
This will be useful for exporting data in non-file formats. For example, can be used to output grib data directly to S3 using the boto3 client without the need to write a temporary file to upload first.
Parameters
- validate (default=True):
If
True
(DEFAULT), validates first/last four bytes for proper formatting, else returns None. IfFalse
, message is output as is.
Returns
- to_bytes: Returns GRIB2 formatted message as bytes.
1288 def interpolate(self, method, grid_def_out, method_options=None, drtn=None, 1289 num_threads=1): 1290 """ 1291 Grib2Message Interpolator 1292 1293 Performs spatial interpolation via the [NCEPLIBS-ip 1294 library](https://github.com/NOAA-EMC/NCEPLIBS-ip). This interpolate 1295 method only supports scalar interpolation. If you need to perform 1296 vector interpolation, use the module-level `grib2io.interpolate` 1297 function. 1298 1299 Parameters 1300 ---------- 1301 method 1302 Interpolate method to use. This can either be an integer or string 1303 using the following mapping: 1304 1305 | Interpolate Scheme | Integer Value | 1306 | :---: | :---: | 1307 | 'bilinear' | 0 | 1308 | 'bicubic' | 1 | 1309 | 'neighbor' | 2 | 1310 | 'budget' | 3 | 1311 | 'spectral' | 4 | 1312 | 'neighbor-budget' | 6 | 1313 1314 grid_def_out : grib2io.Grib2GridDef 1315 Grib2GridDef object of the output grid. 1316 method_options : list of ints, optional 1317 Interpolation options. See the NCEPLIBS-ip documentation for 1318 more information on how these are used. 1319 drtn 1320 Data Representation Template to be used for the returned 1321 interpolated GRIB2 message. When `None`, the data representation 1322 template of the source GRIB2 message is used. Once again, it is the 1323 user's responsibility to properly set the Data Representation 1324 Template attributes. 1325 num_threads : int, optional 1326 Number of OpenMP threads to use for interpolation. The default 1327 value is 1. If NCEPLIBS-ip and grib2io's iplib extension module 1328 was not built with OpenMP, then this keyword argument and value 1329 will have no impact. 1330 1331 Returns 1332 ------- 1333 interpolate 1334 If interpolating to a grid, a new Grib2Message object is returned. 1335 The GRIB2 metadata of the new Grib2Message object is identical to 1336 the input except where required to be different because of the new 1337 grid specs and possibly a new data representation template. 1338 1339 If interpolating to station points, the interpolated data values are 1340 returned as a numpy.ndarray. 1341 """ 1342 section0 = self.section0 1343 section0[-1] = 0 1344 gds = [0, grid_def_out.npoints, 0, 255, grid_def_out.gdtn] 1345 section3 = np.concatenate((gds,grid_def_out.gdt)) 1346 drtn = self.drtn if drtn is None else drtn 1347 1348 msg = Grib2Message(section0,self.section1,self.section2,section3, 1349 self.section4,None,self.bitMapFlag.value,drtn=drtn) 1350 1351 msg._msgnum = -1 1352 msg._deflist = self._deflist 1353 msg._coordlist = self._coordlist 1354 shape = (msg.ny,msg.nx) 1355 ndim = 2 1356 if msg.typeOfValues == 0: 1357 dtype = 'float32' 1358 elif msg.typeOfValues == 1: 1359 dtype = 'int32' 1360 msg._data = interpolate(self.data,method,Grib2GridDef.from_section3(self.section3),grid_def_out, 1361 method_options=method_options,num_threads=num_threads).reshape(msg.ny,msg.nx) 1362 msg.section5[0] = grid_def_out.npoints 1363 return msg
Grib2Message Interpolator
Performs spatial interpolation via the NCEPLIBS-ip
library. This interpolate
method only supports scalar interpolation. If you need to perform
vector interpolation, use the module-level grib2io.interpolate
function.
Parameters
- method: Interpolate method to use. This can either be an integer or string using the following mapping:
Interpolate Scheme | Integer Value |
---|---|
'bilinear' | 0 |
'bicubic' | 1 |
'neighbor' | 2 |
'budget' | 3 |
'spectral' | 4 |
'neighbor-budget' | 6 |
- grid_def_out (grib2io.Grib2GridDef): Grib2GridDef object of the output grid.
- method_options (list of ints, optional): Interpolation options. See the NCEPLIBS-ip documentation for more information on how these are used.
- drtn: Data Representation Template to be used for the returned
interpolated GRIB2 message. When
None
, the data representation template of the source GRIB2 message is used. Once again, it is the user's responsibility to properly set the Data Representation Template attributes. - num_threads (int, optional): Number of OpenMP threads to use for interpolation. The default value is 1. If NCEPLIBS-ip and grib2io's iplib extension module was not built with OpenMP, then this keyword argument and value will have no impact.
Returns
- interpolate: If interpolating to a grid, a new Grib2Message object is returned. The GRIB2 metadata of the new Grib2Message object is identical to the input except where required to be different because of the new grid specs and possibly a new data representation template.
If interpolating to station points, the interpolated data values are returned as a numpy.ndarray.
1366 def subset(self, lats, lons): 1367 """ 1368 Return a spatial subset. 1369 1370 Currently only supports regular grids of the following types: 1371 1372 | Grid Type | gdtn | 1373 | :---: | :---: | 1374 | Latitude/Longitude, Equidistant Cylindrical, or Plate Carree | 0 | 1375 | Rotated Latitude/Longitude | 1 | 1376 | Mercator | 10 | 1377 | Polar Stereographic | 20 | 1378 | Lambert Conformal | 30 | 1379 | Albers Equal-Area | 31 | 1380 | Gaussian Latitude/Longitude | 40 | 1381 | Equatorial Azimuthal Equidistant Projection | 110 | 1382 1383 Parameters 1384 ---------- 1385 lats 1386 List or tuple of latitudes. The minimum and maximum latitudes will 1387 be used to define the southern and northern boundaries. 1388 1389 The order of the latitudes is not important. The function will 1390 determine which is the minimum and maximum. 1391 1392 The latitudes should be in decimal degrees with 0.0 at the equator, 1393 positive values in the northern hemisphere increasing to 90, and 1394 negative values in the southern hemisphere decreasing to -90. 1395 lons 1396 List or tuple of longitudes. The minimum and maximum longitudes 1397 will be used to define the western and eastern boundaries. 1398 1399 The order of the longitudes is not important. The function will 1400 determine which is the minimum and maximum. 1401 1402 GRIB2 longitudes should be in decimal degrees with 0.0 at the prime 1403 meridian, positive values increasing eastward to 360. There are no 1404 negative GRIB2 longitudes. 1405 1406 The typical west longitudes that start at 0.0 at the prime meridian 1407 and decrease to -180 westward, are converted to GRIB2 longitudes by 1408 '360 - (absolute value of the west longitude)' where typical 1409 eastern longitudes are unchanged as GRIB2 longitudes. 1410 1411 Returns 1412 ------- 1413 subset 1414 A spatial subset of a GRIB2 message. 1415 """ 1416 if self.gdtn not in [0, 1, 10, 20, 30, 31, 40, 110]: 1417 raise ValueError( 1418 """ 1419 1420Subset only works for 1421 Latitude/Longitude, Equidistant Cylindrical, or Plate Carree (gdtn=0) 1422 Rotated Latitude/Longitude (gdtn=1) 1423 Mercator (gdtn=10) 1424 Polar Stereographic (gdtn=20) 1425 Lambert Conformal (gdtn=30) 1426 Albers Equal-Area (gdtn=31) 1427 Gaussian Latitude/Longitude (gdtn=40) 1428 Equatorial Azimuthal Equidistant Projection (gdtn=110) 1429 1430""" 1431 ) 1432 1433 if self.nx == 0 or self.ny == 0: 1434 raise ValueError( 1435 """ 1436 1437Subset only works for regular grids. 1438 1439""" 1440 ) 1441 1442 newmsg = Grib2Message( 1443 np.copy(self.section0), 1444 np.copy(self.section1), 1445 np.copy(self.section2), 1446 np.copy(self.section3), 1447 np.copy(self.section4), 1448 np.copy(self.section5), 1449 ) 1450 1451 msglats, msglons = self.grid() 1452 1453 la1 = np.max(lats) 1454 lo1 = np.min(lons) 1455 la2 = np.min(lats) 1456 lo2 = np.max(lons) 1457 1458 # Find the indices of the first and last grid points to the nearest 1459 # lat/lon values in the grid. 1460 first_lat = np.abs(msglats - la1) 1461 first_lon = np.abs(msglons - lo1) 1462 max_idx = np.maximum(first_lat, first_lon) 1463 first_j, first_i = np.where(max_idx == np.min(max_idx)) 1464 1465 last_lat = np.abs(msglats - la2) 1466 last_lon = np.abs(msglons - lo2) 1467 max_idx = np.maximum(last_lat, last_lon) 1468 last_j, last_i = np.where(max_idx == np.min(max_idx)) 1469 1470 setattr(newmsg, "latitudeFirstGridpoint", msglats[first_j[0], first_i[0]]) 1471 setattr(newmsg, "longitudeFirstGridpoint", msglons[first_j[0], first_i[0]]) 1472 setattr(newmsg, "nx", np.abs(first_i[0] - last_i[0])) 1473 setattr(newmsg, "ny", np.abs(first_j[0] - last_j[0])) 1474 1475 # Set *LastGridpoint attributes even if only used for gdtn=[0, 1, 40]. 1476 # This information is used to subset xarray datasets and even though 1477 # unnecessary for some supported grid types, it won't affect a grib2io 1478 # message to set them. 1479 setattr(newmsg, "latitudeLastGridpoint", msglats[last_j[0], last_i[0]]) 1480 setattr(newmsg, "longitudeLastGridpoint", msglons[last_j[0], last_i[0]]) 1481 1482 setattr( 1483 newmsg, 1484 "data", 1485 self.data[ 1486 min(first_j[0], last_j[0]) : max(first_j[0], last_j[0]), 1487 min(first_i[0], last_i[0]) : max(first_i[0], last_i[0]), 1488 ].copy(), 1489 ) 1490 1491 # Need to set the newmsg._sha1_section3 to a blank string so the grid 1492 # method ignores the cached lat/lon values. This will force the grid 1493 # method to recompute the lat/lon values for the subsetted grid. 1494 newmsg._sha1_section3 = "" 1495 newmsg.grid() 1496 1497 return newmsg
Return a spatial subset.
Currently only supports regular grids of the following types:
Grid Type | gdtn |
---|---|
Latitude/Longitude, Equidistant Cylindrical, or Plate Carree | 0 |
Rotated Latitude/Longitude | 1 |
Mercator | 10 |
Polar Stereographic | 20 |
Lambert Conformal | 30 |
Albers Equal-Area | 31 |
Gaussian Latitude/Longitude | 40 |
Equatorial Azimuthal Equidistant Projection | 110 |
Parameters
- lats: List or tuple of latitudes. The minimum and maximum latitudes will be used to define the southern and northern boundaries.
The order of the latitudes is not important. The function will determine which is the minimum and maximum.
The latitudes should be in decimal degrees with 0.0 at the equator, positive values in the northern hemisphere increasing to 90, and negative values in the southern hemisphere decreasing to -90.
- lons: List or tuple of longitudes. The minimum and maximum longitudes will be used to define the western and eastern boundaries.
The order of the longitudes is not important. The function will determine which is the minimum and maximum.
GRIB2 longitudes should be in decimal degrees with 0.0 at the prime meridian, positive values increasing eastward to 360. There are no negative GRIB2 longitudes.
The typical west longitudes that start at 0.0 at the prime meridian and decrease to -180 westward, are converted to GRIB2 longitudes by '360 - (absolute value of the west longitude)' where typical eastern longitudes are unchanged as GRIB2 longitudes.
Returns
- subset: A spatial subset of a GRIB2 message.
1499 def validate(self): 1500 """ 1501 Validate a complete GRIB2 message. 1502 1503 The g2c library does its own internal validation when g2_gribend() is called, but 1504 we will check in grib2io also. The validation checks if the first 4 bytes in 1505 self._msg is 'GRIB' and '7777' as the last 4 bytes and that the message length in 1506 section 0 equals the length of the packed message. 1507 1508 Returns 1509 ------- 1510 `True` if the packed GRIB2 message is complete and well-formed, `False` otherwise. 1511 """ 1512 valid = False 1513 if hasattr(self,'_msg'): 1514 if self._msg[0:4]+self._msg[-4:] == b'GRIB7777': 1515 if self.section0[-1] == len(self._msg): 1516 valid = True 1517 return valid
Validate a complete GRIB2 message.
The g2c library does its own internal validation when g2_gribend() is called, but we will check in grib2io also. The validation checks if the first 4 bytes in self._msg is 'GRIB' and '7777' as the last 4 bytes and that the message length in section 0 equals the length of the packed message.
Returns
True
if the packed GRIB2 message is complete and well-formed,False
otherwise.
1930@dataclass 1931class Grib2GridDef: 1932 """ 1933 Class for Grid Definition Template Number and Template as attributes. 1934 1935 This allows for cleaner looking code when passing these metadata around. 1936 For example, the `grib2io._Grib2Message.interpolate` method and 1937 `grib2io.interpolate` function accepts these objects. 1938 """ 1939 gdtn: int 1940 gdt: NDArray 1941 1942 @classmethod 1943 def from_section3(cls, section3): 1944 return cls(section3[4],section3[5:]) 1945 1946 @property 1947 def nx(self): 1948 """Number of grid points in x-direction.""" 1949 return self.gdt[7] 1950 1951 @property 1952 def ny(self): 1953 """Number of grid points in y-direction.""" 1954 return self.gdt[8] 1955 1956 @property 1957 def npoints(self): 1958 """Total number of grid points.""" 1959 return self.gdt[7] * self.gdt[8] 1960 1961 @property 1962 def shape(self): 1963 """Shape of the grid.""" 1964 return (int(self.ny), int(self.nx))
Class for Grid Definition Template Number and Template as attributes.
This allows for cleaner looking code when passing these metadata around.
For example, the grib2io._Grib2Message.interpolate
method and
grib2io.interpolate
function accepts these objects.
1946 @property 1947 def nx(self): 1948 """Number of grid points in x-direction.""" 1949 return self.gdt[7]
Number of grid points in x-direction.
1951 @property 1952 def ny(self): 1953 """Number of grid points in y-direction.""" 1954 return self.gdt[8]
Number of grid points in y-direction.