These are my notes for how logging works for Python applications on Google App Engine and how it integrates with Cloud Logging.

I used the logging_tree package to understand how Python’s logging system gets initialized in different scenarios. Read Brandon Rhodes’ article introducing logging_tree.

Logging on the Python 2.7 runtime

• Python’s logging system is automatically configured to use the handler in the google.appengine.api.logservice package from the SDK. No code is required in your application to enable this integration.
• The default logging level is DEBUG.
• Messages are buffered for a maximum of 60 seconds.
• The log name is "projects/[PROJECT-ID]/logs/appengine.googleapis.com%2Frequest_log".
• Messages can have multiple lines (records), which can be used by the handler to combine more than one log record.

Logging on the Python 3.9 runtime (default setup)

• No automatic configuration for Python logging.
• The default handler is the STDERR handler.
• The default logging level is WARNING (this is the default level for Python’s logging module).
• App Engine reads the application output from STDERR and sends it to Cloud Logging.
• The log name is "projects/[PROJECT-ID]/logs/stderr".
• Messages are created with a textPayload but no other structured information from the Python logging – message.

Logging with google-cloud-logging on the Python 3.9 runtime

• Add the google-cloud-logging package to "requirements.txt".
• Enable it with import google.cloud.logging; google.cloud.logging.Client().setup_logging().
• The default logging level is INFO.
• Use setup_logging(log_level=logging.DEBUG) to set a DEBUG level.
• The log name is "projects/[PROJECT-ID]/logs/app".
• Messages are created with a "jsonPayload" and with the correct log level (the "severity" field in log records).
• If Flask is installed, the logging handler gets the trace ID from the request.
• If Django is installed, and you enabled google.cloud.logging.handlers.middleware.RequestMiddleware, the logging handler gets the trace ID from the request.

Logging for applications that don’t use Flask or Django

For Python applications on App Engine, the important thing is to enable the AppEngineHandler logging handler provided by the google-cloud-logging package when the application starts:

# main.py
import google.cloud.logging

google.cloud.logging.Client().setup_logging()

This will give you log messages such that you can filter by level in the logs explorer.

However if your application does not use Flask or Django, log messages will not have the request’s trace ID, and that makes it harder to identify which messages are associated with a request.

I wrote a demo application with a logging handler to use with the Bottle web framework. The handler adds the correct logging trace ID and other request information.

It does not take much code to extend the logging system this way, but reading the source code for google-cloud-logging reminded me that sending messages to the logging system has an overhead and can only make your application a little slower. Make sure to use the logging API features that avoid unnecessary work (loggers, levels and positional message arguments) and even better just don’t log a message unless you know you will need it to debug a problem later.