Experience Report for Feature Request
Currently logging in practice with dgraph logging is limited to sending to stdout or stderr.
What you wanted to do
This is an enhancement request to have some these configurable options, such as:
- This is an enhancement request to have some configurable routing to direct logs to configurable paths for standard logs, error logs, query logs, debug logs, and access logs.
- Ability route logs to something other than files or stdout/stderr, such as syslog.
- The logging format should be configurable by the user, such as using a
- Tailing timestamp format
- Error and Warnings should by default go to stderr and indicate they are an error or warning. In some cases, there are errors reported as info.
What you actually did
You need an external system to apply advanced regex patterns to filter and route logs, do compaction, retention, and rotation.
Why that wasn’t great, with examples
Currently, there’s a high barrier to use Dgraph in production systems, especially for enterprise features like ACL, where logging can quickly overload the disk and choke alpha node of disk resources.
This would be acceptable if there was an enterprise feature, e.g. logging vs. open-source logging, with exception of item 5 (errors should be flag as
Any external references to support your case
Many enterprise systems have the notion of having a separate error log and other logs, and a few have authorization logs for tacking logins events. Systems that face access through web endpoints, will also have access logs, as this is useful for security or discovery types of users accessing the system.
configuration file support
systemLog.destinationoption for file or syslog
systemLog.timeStampFormatsupported in configuration with default of
- ref. https://docs.mongodb.com/manual/reference/log-messages/
- logs segregated into
- logs have expected categories of
ERROR, and also
- security and query logging are enterprise features.
- ref. https://neo4j.com/docs/operations-manual/current/monitoring/logging/
- Uses SLF4J (Simple Logging Facade for Java) with logback backend
- log file retention, log file rotation, extended compaction logging
- ref. https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/configuration/configLoggingLevels.html
- logs have categories of
ERROR, and special highest severity of
WARNINGand above go to stderr and write to a file.
- have query logs that are configurable, such as
- have authentication logs, audit logs
- log locations are configurable.
- ref. https://www.cockroachlabs.com/docs/stable/debug-and-error-logs.html
- ref. https://www.cockroachlabs.com/docs/v20.1/query-behavior-troubleshooting
- ref. https://www.cockroachlabs.com/docs/v20.1/cockroach-start#logging