PySMI library¶

The MibCompiler object is the top-most interface to PySMI library features. It holds together the otherwise isolated pieces of the compiler infrastructure and manages the workflow of ASN.1 MIB transformation.

This example showcases some of its features:

from pysmi.reader import HttpReader
from pysmi.searcher import StubSearcher
from pysmi.writer import CallbackWriter
from pysmi.parser import SmiStarParser
from pysmi.codegen import JsonCodeGen
from pysmi.compiler import MibCompiler

inputMibs = ['IF-MIB', 'IP-MIB']

httpSources = ['https://pysnmp.github.io/mibs/asn1/@mib@']

# store compiled MIBs by calling this function
def store_mibs(mibName, jsonDoc, cbCtx):
    print('# MIB module %s' % mibName)
    print(jsonDoc)

mibCompiler = MibCompiler(
    SmiStarParser(), JsonCodeGen(), CallbackWriter(store_mibs)
)

# pull ASN.1 MIBs over HTTP
mibCompiler.add_sources(*[HttpReader(x) for x in httpSources])

# never recompile MIBs with ASN.1 MACROs
mibCompiler.add_searchers(StubSearcher(*JsonCodeGen.baseMibs))

status = mibCompiler.compile(*inputMibs)

print(status)

Method naming¶

PySMI’s methods were camelCase – addSources, getMibVariants, putData and so on. They are snake_case now, as PEP 8 calls for: add_sources, get_mib_variants, put_data.

Every renamed method keeps its old name, so existing code goes on working:

mibCompiler.addSources(HttpReader(url))   # works, warns
mibCompiler.add_sources(HttpReader(url))  # the same thing, quietly

Calling the old name raises a DeprecationWarning, which Python hides by default. To see where your code still uses the old spelling, run it with warnings turned on:

python -W always::DeprecationWarning your_script.py

The old names will be removed in a future major release.

If you subclass a PySMI class and override a method under its old name, PySMI installs your override under the new name and warns, so it still runs. Renaming the override silences the warning.

MIB sources¶

PySMI offers a handful of distinct transport mechanisms for fetching MIBs by name from specific locations. In all cases MIB module name to file name match may not be exact – some name fuzzying can be performed to mitigate possible changes to MIB file name.

Parse caches¶

Parsing ASN.1 is the expensive part of a compile, and a build over many source sets presents the same standard modules again for every set. A parse cache holds the trees so that text is parsed once per build rather than once per set; see Compiling many source sets for the pattern.

The provider is a component like any other: pass one to MibCompiler, or write your own against AbstractParseCache. The compiler uses the object it is handed and never resolves a provider by name, from an entry point or from configuration, so the trust boundary is your own code – which matters, because a provider that stores trees outside the process reconstructs arbitrary Python objects when it reads them back.

Conditional compilation¶

There are cases when MIB transformation may or must not be performed. Such cases include:

foundation MIBs containing manually implemented pieces or ASN.1 MACRO’s
obsolete MIBs fully reimplemented within modern MIBs
already transformed MIBs

MibCompiler expects user to supply a searcher object that would allow or skip MIB transformation for particular name based on whatever reason it is aware of.

In general, searcher logic is specific to target format. At the time being, only pysnmp code generation backend requires such filtering.

Parser configuration¶

MIBs may be written in one of the two major SMI language versions (v1 and v2). Some MIBs may contain typical errors.

PySMI offers a way to customize the parser to consume either of the major SMI grammars as well as to recover from well-known errors in MIB files.

Code generators¶

Once ASN.1 MIB is parsed up, AST is passed to a code generator which turns AST into desired representation of the MIB.

Borrow pre-compiled MIBs¶

Some MIBs in circulation appear broken beyond automatic repair. To handle such cases PySMI introduces the MIB borrowing functionality. When MibCompiler gives up compiling a MIB, it can try to go out and take a copy of already transformed MIB to complete the request successfully.

Write compiled MIBs¶

Successfully transformed MIB modules’ contents will be passed to writer object given to MibCompiler on instantiation.

Extending PySMI¶

Each stage of the compiler is defined by a small interface, so a MIB source, searcher, parser, code generator or writer of your own can be dropped in beside the ones PySMI ships.

Extension points

Bundled MIBs¶

The modules PySMI bundles carry a manifest recording where each one’s text comes from and, for those a later RFC replaced, which module took over.

Bundled MIB manifest
- Carried and held

Exceptions¶

Every stage reports failure by raising an exception of its own, all of them derived from a single base class.

Exceptions

Examples¶

The following examples focus on various feature of the PySMI library.

In case of any troubles or confusion, try enabling PySMI debugging and watch the output:

from pysmi import debug

debug.enableDebugLogging('all')

Pass the names of the subsystems you are interested in to keep the output down, prefixing a name with ! to leave that one out:

debug.enableDebugLogging('reader', 'compiler')
debug.enableDebugLogging('all', '!grammar')

PySMI logs through the standard logging module, and each subsystem logs to the logger named after its package, so an application that already configures logging can select and route this output itself without going through PySMI at all:

import logging

logging.getLogger('pysmi.compiler').setLevel(logging.DEBUG)

Messages carry their variable parts as structured fields on the log record – the name of the MIB being worked on as mib, the file being read or written as path, and so on – so a handler can pick them out individually:

class MibHandler(logging.Handler):
    def emit(self, record):
        print(record.getMessage(), getattr(record, 'mib', None))

Note

debug.setLogger() and debug.Debug() still work, but are deprecated in favour of debug.enableDebugLogging().