loglama

LogLama Scripts

This directory contains scripts to help integrate LogLama into existing PyLama ecosystem components, migrate projects from various logging systems to LogLama, and fix linting and syntax issues in the codebase.

integrate_loglama.py

This script helps you integrate LogLama into any component of the PyLama ecosystem. It will:

1. Create a logging_config.py module in the target component
2. Update .env and .env.example files with LogLama configuration
3. Create necessary directories (logs, etc.)

Usage

# Integrate LogLama into a specific component
./integrate_loglama.py --component apilama

# Integrate LogLama into all components in the PyLama ecosystem
./integrate_loglama.py --all

# Specify a custom base directory
./integrate_loglama.py --all --dir /path/to/py-lama

# Generate a report without making any changes
./integrate_loglama.py --all --report-only

Integration Status Report

The script can generate a comprehensive report of the LogLama integration status across all components using the --report-only flag. This is useful for checking which components have already been integrated and which ones still need work.

The report includes the following information for each component:

• Whether the component has a logging_config.py file
• Whether the component has a logs directory
• Whether the component’s .env file includes LogLama configuration
• Whether the component’s .env.example file includes LogLama configuration
• Overall integration status (fully integrated or not)

Example report output:

=== LogLama Integration Report ===

Component Status:

apilama         | ✅ Fully integrated
bexy           | ✅ Fully integrated
getllm           | ✅ Fully integrated
devlama          | ✅ Fully integrated
shellama        | ✅ Fully integrated
weblama         | ✅ Fully integrated

Integration Steps

After running the script, you’ll need to update your component’s main module to use LogLama. Add the following at the very top of your main module (before any other imports):

# Initialize logging first, before any other imports
from your_component.logging_config import init_logging, get_logger

# Initialize logging with LogLama
init_logging()

# Now import other standard libraries
import os
import sys
# ... other imports

# Get a logger
logger = get_logger('your_module')

Integrated Components

The following PyLama ecosystem components have been integrated with LogLama:

1. APILama: REST API gateway for the PyLama ecosystem
2. BEXY: Sandbox environment for executing code
3. PyLLM: LLM service for code generation
4. PyLama: Core component for the PyLama ecosystem
5. ShellLama: Shell command execution service
6. WebLama: Web interface for the PyLama ecosystem

You can check the integration status of each component using the --report-only flag as described above.

Benefits of Using LogLama

1. Early Environment Loading: LogLama loads environment variables from .env files before any other libraries, preventing issues with incorrect configuration.

2. Structured Logging: LogLama provides structured logging capabilities, making it easier to search and analyze logs.

3. Multiple Outputs: Logs can be sent to console, files, SQLite database, and more.

4. Context-Aware Logging: Add context to your logs for better debugging.

5. Consistent Interface: All components use the same logging interface, making it easier to understand and maintain.

6. Web Interface: Access and filter logs through a convenient web interface using the loglama web command.

Configuration

LogLama can be configured through environment variables in your .env file:

# LogLama configuration
COMPONENT_LOG_LEVEL=INFO                # Options: DEBUG, INFO, WARNING, ERROR, CRITICAL
COMPONENT_LOG_DIR=./logs               # Directory to store log files
COMPONENT_DB_LOGGING=true              # Enable database logging for advanced querying
COMPONENT_DB_PATH=./logs/component.db  # Path to SQLite database for logs
COMPONENT_JSON_LOGS=false              # Use JSON format for logs (useful for log processors)

# LogLama advanced settings
LOGLAMA_STRUCTURED_LOGGING=false        # Use structured logging with structlog
LOGLAMA_MAX_LOG_SIZE=10485760           # Maximum log file size in bytes (10 MB)
LOGLAMA_BACKUP_COUNT=5                  # Number of backup log files to keep

Replace COMPONENT with your component name (e.g., APILAMA, WEBLAMA, etc.).

migrate_to_loglama.py

This script helps you migrate projects from LogLama to LogLama. It will automatically update import statements, variable names, function calls, environment variables, and file names.

Usage

# Run in report-only mode to see what would change without making any changes
python migrate_to_loglama.py --path /path/to/your/project --report-only --verbose

# Run for real to make the changes
python migrate_to_loglama.py --path /path/to/your/project --verbose

# Specify a custom output file for the migration report
python migrate_to_loglama.py --path /path/to/your/project --output migration_results.json

Linting and Syntax Fixing Scripts

The following scripts are designed to fix various linting and syntax issues in the LogLama codebase. They are organized by purpose and severity.

fix_lint.py

This script automatically fixes common linting issues in the LogLama codebase.

Fixes:

• Unused imports (using autoflake)
• Lines exceeding 79 characters (using black with line-length=79)
• Blank lines with whitespace (using black)
• F-strings missing placeholders (manual fix)
• Indentation issues (using black)
• Spacing between functions (using black)
• Trailing whitespace (using black)

Usage:

python scripts/fix_lint.py

fix_advanced_lint.py

This script addresses more complex linting issues that require more sophisticated fixes.

Fixes:

• Lines exceeding 79 characters (using manual line breaking)
• Unused variables (F841)
• Imports not at the top of files (E402)
• Membership tests (E713)

Usage:

python scripts/fix_advanced_lint.py

fix_basic_syntax.py

This script focuses on the most critical syntax issues that are preventing the package from being published.

Fixes:

• Unmatched parentheses
• Missing colons after function definitions
• Indentation after colons
• Adding noqa comments to long lines

Usage:

python scripts/fix_basic_syntax.py

add_noqa.py

This script adds # noqa comments to lines with linting issues to allow the package to pass linting checks without fixing all errors immediately.

Fixes:

• Adds # noqa comments to lines with linting issues
• Adds specific error codes to noqa comments when possible

Usage:

python scripts/add_noqa.py

fix_publish.py

This comprehensive script fixes critical issues blocking the publish process in LogLama.

Fixes:

• Docstring format issues
• Unterminated triple-quoted strings
• Unused imports (by adding # noqa comments)
• Long lines (by adding # noqa: E501 comments)
• Syntax errors in except blocks
• Creates empty init.py files in missing directories

Usage:

python scripts/fix_publish.py

fix_docstrings.py

This script specifically targets docstring-related issues in the codebase.

Fixes:

• Unterminated docstrings
• Malformed docstrings with extra quotes
• Incorrect docstring indentation
• Except blocks with syntax errors
• Unmatched parentheses

Usage:

python scripts/fix_docstrings.py

fix_syntax.py and fix_syntax_v2.py

These scripts address specific syntax issues that are preventing the package from being published. The v2 version contains improvements over the original.

Fixes:

• Unmatched parentheses
• Indentation errors
• String literal syntax errors
• Unterminated strings
• Missing colons

Usage:

python scripts/fix_syntax_v2.py  # Recommended version

fix_critical_lint.py

This script addresses critical linting issues that should be run after the other linting scripts.

Fixes:

• Undefined variables in except blocks (F821)
• Long lines (E501) using manual line breaking for specific files
• Syntax errors in string literals
• Redefined imports

Usage:

python scripts/fix_critical_lint.py

Recommended Usage Order

For the most effective linting and syntax fixing, run the scripts in the following order:

1. fix_basic_syntax.py - Fix critical syntax errors first
2. fix_docstrings.py - Fix docstring issues
3. fix_syntax_v2.py - Fix remaining syntax issues
4. fix_lint.py - Fix common linting issues
5. fix_advanced_lint.py - Fix more complex linting issues
6. fix_critical_lint.py - Fix critical remaining issues
7. add_noqa.py - Add noqa comments to remaining issues
8. fix_publish.py - Final comprehensive fixes for publishing

Alternatively, you can run just fix_publish.py for a quick solution that focuses on making the minimal necessary changes to allow the package to pass linting checks for publication.

What It Does

The migration script will:

1. Update import statements from import loglama to import loglama
2. Update module references from loglama.xyz to loglama.xyz
3. Update variable names containing loglama to use loglama instead
4. Update function calls like loglama_get_logger() to loglama_get_logger()
5. Update logger names (e.g., from “loglama_examples” to “loglama_examples”)
6. Update environment variable references from LOGLAMA_ to LOGLAMA_
7. Update configuration file names from loglama_config.json to loglama_config.json
8. Rename files and directories containing loglama in their names

Migration Guide

For a comprehensive migration guide, see the MIGRATION_GUIDE.md file in this directory.

Compatibility Layer

If you need to maintain compatibility with both LogLama and LogLama during a transition period, you can use the compatibility layer provided in loglama.compat.

# Instead of importing directly from loglama or loglama
from loglama.compat import get_logger, setup_logging, LogContext

# Then use as normal
logger = get_logger(__name__)
logger.info("This works with both LogLama and LogLama!")

universal_log_migrator.py

This script provides a comprehensive solution for migrating projects from various logging systems to LogLama. It supports multiple source logging systems and provides a unified interface for migration.

Supported Logging Systems

• Standard library logging: Python’s built-in logging module
• Loguru: A popular third-party logging library
• structlog: A structured logging library
• Custom logging implementations: Detects and migrates common custom logging patterns
• LogLama: Uses the migrate_to_loglama.py script for LogLama migration

Usage

# Migrate from Python's standard logging module
python universal_log_migrator.py --path /path/to/your/project --source logging --verbose

# Migrate from Loguru
python universal_log_migrator.py --path /path/to/your/project --source loguru

# Migrate from structlog
python universal_log_migrator.py --path /path/to/your/project --source structlog

# Migrate from a custom logging implementation
python universal_log_migrator.py --path /path/to/your/project --source custom

# Migrate from LogLama
python universal_log_migrator.py --path /path/to/your/project --source loglama

# Run in report-only mode to see what would change without making any changes
python universal_log_migrator.py --path /path/to/your/project --source logging --report-only

What It Does

The universal log migrator will:

1. Scan your project for files containing references to the source logging system
2. Update import statements to use LogLama
3. Convert logging configuration to use LogLama’s setup_logging()
4. Update log method calls (debug, info, warning, error, critical)
5. Convert context handling to use LogLama’s LogContext
6. Rename files and directories related to logging
7. Create a default logging_config.py if none exists
8. Update requirements.txt or pyproject.toml to include LogLama

Migration Report

After running the migrator, a detailed report is generated showing all changes made. This report includes:

• Files modified
• Lines changed
• Types of changes (imports, configuration, log calls, etc.)
• Files and directories renamed
• New files created

The report is saved as JSON for easy parsing and can be viewed with any text editor or JSON viewer.

This site is open source. Improve this page.