Skip to main content

Logging Utility for ML Experiments

Project description

CircleCI PyPI - License PyPI - Python Version Code style: black

ml-logger

Logging utility for ML experiments

Installation

  • pip install "mllogger[all]==0.3"

If you want to use only the filesystem logger, use pip install "mllogger==0.3"

Install from source

  • git clone git@github.com:shagunsodhani/ml-logger.git
  • cd ml-logger
  • pip install ".[all]"

Alternatively, pip install "git+https://git@github.com/shagunsodhani/ml-logger.git@master#egg=ml_logger[all]"

If you want to use only the filesystem logger, use pip install . or pip install "git+https://git@github.com/shagunsodhani/ml-logger.git@master#egg=ml_logger".

Documentation

https://shagunsodhani.github.io/ml-logger

Use

  • Make a logbook_config:

    from ml_logger import logbook as ml_logbook
    logbook_config = ml_logbook.make_config(
        logger_file_path = <path to write logs>,
        wandb_config = <wandb config or None>,
        tensorboard_config = <tensorboard config or None>,
        mlflow_config = <mlflow config or None>)
    

    The API for make_config can be accessed here.

  • Make a LogBook instance:

    logbook = ml_logbook.LogBook(config = logbook_config)
    
  • Use the logbook instance:

    log = {
        "epoch": 1,
        "loss": 0.1,
        "accuracy": 0.2
    }
    logbook.write_metric_log(log)
    

    The API for write_metric_log can be accessed here.

Note

  • If you are writing to wandb, the log must have a key called step. If your log already captures the step but as a different key (say epoch), you can pass the wandb_key_map argument (set as {epoch: step}). For more details, refer the documentation here.

  • If you are writing to mlflow, the log must have a key called step. If your log already captures the step but as a different key (say epoch), you can pass the mlflow_key_map argument (set as {epoch: step}). For more details, refer the documentation here.

  • If you are writing to tensorboard, the log must have a key called main_tag or tag which acts as the data Identifier and another key called global_step. These keys are described here. If your log already captures these values but as different key (say mode for main_tag and epoch for global_step), you can pass the tensorboard_key_map argument (set as {mode: main_tag, epoch: global_step}). For more details, refer the documentation here.

Dev Setup

  • pip install -e ".[dev]"
  • Install pre-commit hooks pre-commit install
  • The code is linted using:
    • black
    • flake8
    • mypy
  • CI currently only checks for linting.
  • Lint tests can be run locally using nox -s lint

Acknowledgements

  • Config for circleci, pre-commit, mypy etc are borrowed/modified from Hydra

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page