Experience Report for Feature Request
Currently logging in practice with dgraph logging is limited to sending to stdout or stderr.
What you wanted to do
This is an enhancement request to have some these configurable options, such as:
- This is an enhancement request to have some configurable routing to direct logs to configurable paths for standard logs, error logs, query logs, debug logs, and access logs.
- Ability route logs to something other than files or stdout/stderr, such as syslog.
- The logging format should be configurable by the user, such as using a
JSONFormatterorTextFormatter. - Tailing timestamp format
- Error and Warnings should by default go to stderr and indicate they are an error or warning. In some cases, there are errors reported as info.
What you actually did
You need an external system to apply advanced regex patterns to filter and route logs, do compaction, retention, and rotation.
Why that wasn’t great, with examples
Currently, there’s a high barrier to use Dgraph in production systems, especially for enterprise features like ACL, where logging can quickly overload the disk and choke alpha node of disk resources.
This would be acceptable if there was an enterprise feature, e.g. logging vs. open-source logging, with exception of item 5 (errors should be flag as E not I).
Any external references to support your case
Many enterprise systems have the notion of having a separate error log and other logs, and a few have authorization logs for tacking logins events. Systems that face access through web endpoints, will also have access logs, as this is useful for security or discovery types of users accessing the system.
MongoDB
-
configuration file support
systemLog.destinationoption for file or syslog -
commandline suports
--logpath,--syslogformongodandmongos - the
systemLog.timeStampFormatsupported in configuration with default ofiso8601-local. - ref. Log Messages — MongoDB Manual
Neo4J
- logs segregated into
neo4j.log,debug.log,http.log, gc.log,query.log,security.log,service-error.log` - logs have expected categories of
INFO,WARN,ERROR, and alsoDEBUG. - security and query logging are enterprise features.
- ref. Logging - Operations Manual
Cassandra
- Uses SLF4J (Simple Logging Facade for Java) with logback backend
- log file retention, log file rotation, extended compaction logging
- ref. Configuring logging
CockroachDB
- logs have categories of
INFO,WARNING,ERROR, and special highest severity ofFATA. -
WARNINGand above go to stderr and write to a file. - have query logs that are configurable, such as
sql.log.slow_query.latency_threshold - have authentication logs, audit logs
- log locations are configurable.
- ref. Understand Debug & Error Logs | CockroachDB Docs
- ref. Troubleshoot SQL Behavior | CockroachDB Docs
- ref. cockroach start | CockroachDB Docs