From 242127d5284ed7711afda74b51b5bc6648c18b36 Mon Sep 17 00:00:00 2001 From: boyska Date: Tue, 5 Feb 2019 17:41:05 +0100 Subject: [PATCH 1/2] docs with mkdocs --- docs/.gitignore | 1 + docs/docs/hacking.md | 15 +++++++++++ docs/docs/index.md | 54 ++++++++++++++++++++++++++++++++++++++++ docs/docs/install.md | 25 +++++++++++++++++++ docs/{ => docs}/query.md | 0 docs/docs/systemd.md | 29 +++++++++++++++++++++ docs/mkdocs.yml | 8 ++++++ 7 files changed, 132 insertions(+) create mode 100644 docs/.gitignore create mode 100644 docs/docs/hacking.md create mode 100644 docs/docs/index.md create mode 100644 docs/docs/install.md rename docs/{ => docs}/query.md (100%) create mode 100644 docs/docs/systemd.md create mode 100644 docs/mkdocs.yml diff --git a/docs/.gitignore b/docs/.gitignore new file mode 100644 index 0000000..1320f90 --- /dev/null +++ b/docs/.gitignore @@ -0,0 +1 @@ +site diff --git a/docs/docs/hacking.md b/docs/docs/hacking.md new file mode 100644 index 0000000..a55cd3b --- /dev/null +++ b/docs/docs/hacking.md @@ -0,0 +1,15 @@ +## Why not on github.com + +We love collaboration between people, and software developement can be a wonderful playground. Nevertheless, we are not comfortable with the social-neworkish nature of github, and also the choice of a platform is something that is meaningful in itself. +Fortunately go allows to pull packages from any git server that is publicly available. Therefore do not be afraid of go-getting from git.lattuga.net. + +[Spiegare meglio la natura autogestita di lattuga] + +### So how to collaborate + +If you want to collaborate to the software, either drop a mail to or open an account on git.lattuga.net to open a PR. +We are also reachable at [IRC irc.mufhd0.net] + + + + diff --git a/docs/docs/index.md b/docs/docs/index.md new file mode 100644 index 0000000..8c45871 --- /dev/null +++ b/docs/docs/index.md @@ -0,0 +1,54 @@ +# Circolog + +Circolog proposes a differnt approach to the problem of having useful logs. + +Circolog is a syslog server which doesn't write to disk. Ever. It doesn't consume tons of RAM like elasticsearch/logstash, nor does it give you plots and a very long history. +It is however a very useful tool when you want to minimize disk writes. We wrote it thinking about user privacy: your logs can be just as sensitive as your database if you log too much! + +[Spiegone su casi d'uso, buttare i log, debuggare rapidamente magari applicazioni verbose, eccetera] + +## Why it is cool in 2 minutes + +Keep your users safer + +Read logs conveniently: + filter with a proper (and easy) query language! + + grep is powerful, but sometimes you want something more expressive: + + circolog-tail -where 'app_name=="apache" and message LIKE "%memory%"' + + circolog-tail -where '(app_name"apache" OR app_name LIKE "php%") AND message LIKE "%memory%"' + + circolog-tail -where 'app_name=="mysql" OR severity >= warning' + + colors to highlight high severity and visually group related message + + [screenshot] + + It's not only about being nice, we swear! coloring logs also means reaching the most important entries easily, and grouping related entries together. + + + output logs in the format you prefer _now_ + + depending on what you're doing, log format might be useful... or distracting. For example, how many times have you used the `hostname` part of it? With the common disk-based logging, you need to choose once and for all how your logs will be saved. With circolog + + +Hackable + + We think that circolog-tail is pretty cool, but you definitely can reuse simpler tools to get logs and filter them the way you prefer. clients can read logs using plain HTTP (or websocket). Most of the cool features of circolog-tail are actually implemented server-side, so you can use filters (or other options) wih any client. + + +Fast, secure by default (?), easy deploy + + Those are features that you should expect, not be surprised of! circolog can easily process thousands of log entries per seconds, has sane defaults and can be deployed as a single binary + + +## Security considerations + +While we try our best not to introduce vulnerabilities, this software is not meant to be exposed on the wider internet. +Beware of binding it on something different from localhost. + +Even without being exposed, care must be given to socket permissions: don't let unprivileged users read your logs! +We suggest that you use a dedicated user/group to run circolog, and make root part of that group. + diff --git a/docs/docs/install.md b/docs/docs/install.md new file mode 100644 index 0000000..0e4349e --- /dev/null +++ b/docs/docs/install.md @@ -0,0 +1,25 @@ +## Installation and Configuration + + +If you trust us, just get the compiled binaries and copy them to /usr/local/{s,}bin/ (but you shouldn't: [compiling -> altra pagina]). + +Running circologd is pretty easy: just run `circologd` by hand or in your favourite service manager: [here are some contrib scripts] for sysV, systemd, supervisor... but consider them as hints, don't just copy them! + +### System Integration + +While circologd can be your only syslog daemon, you might want to couple it with another syslog to have the best of both worlds. For example, you could use rsyslog (or syslog-ng) to write important (ie: priority >= notice) log to persistent storage, letting circolog handle short-term but heavily detailed logs. + +Or you could make circolog get messages from journald. While possibilities are endless, we tried to document some common setups. + +#### Use Case: circologd as the only syslog + +[mica lo so se funziona, credo di no, ma proviamo] +bind it to /dev/log + +#### Use Case: circologd receiving messages from syslog-ng + +#### Use Case: circologd receiving messages from journald + +[TODO: il codice va ancora scritto: bisogna bindare /run/systemd/journal/syslog con formato != rfc5424 + + diff --git a/docs/query.md b/docs/docs/query.md similarity index 100% rename from docs/query.md rename to docs/docs/query.md diff --git a/docs/docs/systemd.md b/docs/docs/systemd.md new file mode 100644 index 0000000..8d7c783 --- /dev/null +++ b/docs/docs/systemd.md @@ -0,0 +1,29 @@ +unit systemd con utente dinamico: + - no adduser/altre conf + - utente con pochi permessi + - accesso consentito al gruppo adm + + [Unit] + Description=In-memory logging + + [Service] + DynamicUser=true + Group=adm + RuntimeDirectory=circolog + # this is important: circologd will respect umask, so if you want to have files that are not world-readable, you must set it + RuntimeDirectoryMode=0750 + UMask=0026 + ProtectSystem=full + ExecStart=/usr/local/sbin/circologd -syslogd-socket /run/circolog/syslog.sock -buffer-size 2000 -query-socket /run/circolog/query.sock + # security restrictions; useful, but not needed + PrivateTmp=true + PrivateNetwork=true + NoNewPrivileges=true + Restrictnamespaces=true + + #optional: watchdog + WatchdogSec=30 + + [Install] + WantedBy=multi-user.target + diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml new file mode 100644 index 0000000..a60680b --- /dev/null +++ b/docs/mkdocs.yml @@ -0,0 +1,8 @@ +site_name: Circolog +nav: + - Home: index.md + - Install: install.md + - Queries: query.md + - Hacking: hacking.md +repo_url: https://git.lattuga.net/boyska/circolog +repo_name: 'Repository' From 2241d18fa9cc591c309b5d0e87cfe9e92040f8fe Mon Sep 17 00:00:00 2001 From: Blallo Date: Wed, 6 Feb 2019 11:19:31 +0100 Subject: [PATCH 2/2] Docs: fixup format and minor typos. --- docs/docs/hacking.md | 17 ++++++----- docs/docs/index.md | 71 ++++++++++++++++++++++++-------------------- docs/docs/install.md | 24 ++++++++++----- docs/docs/query.md | 15 ++++++---- 4 files changed, 75 insertions(+), 52 deletions(-) diff --git a/docs/docs/hacking.md b/docs/docs/hacking.md index a55cd3b..12a8a74 100644 --- a/docs/docs/hacking.md +++ b/docs/docs/hacking.md @@ -1,15 +1,16 @@ ## Why not on github.com -We love collaboration between people, and software developement can be a wonderful playground. Nevertheless, we are not comfortable with the social-neworkish nature of github, and also the choice of a platform is something that is meaningful in itself. -Fortunately go allows to pull packages from any git server that is publicly available. Therefore do not be afraid of go-getting from git.lattuga.net. +We love collaboration between people, and software development can be +a wonderful playground. Nevertheless, we are not comfortable with the +social-neworkish nature of github. Also, the choice of a platform is something +that is meaningful in itself. Fortunately `go` allows to pull packages from any +git server that is publicly available. Therefore do not be afraid of go-getting +from git.lattuga.net. [Spiegare meglio la natura autogestita di lattuga] ### So how to collaborate -If you want to collaborate to the software, either drop a mail to or open an account on git.lattuga.net to open a PR. -We are also reachable at [IRC irc.mufhd0.net] - - - - +If you want to collaborate to the software, either drop a mail to + or open an account on git.lattuga.net to open a PR. We +are also reachable at [IRC irc.mufhd0.net] diff --git a/docs/docs/index.md b/docs/docs/index.md index 8c45871..567fbd9 100644 --- a/docs/docs/index.md +++ b/docs/docs/index.md @@ -1,54 +1,61 @@ # Circolog -Circolog proposes a differnt approach to the problem of having useful logs. +Circolog proposes a different approach to the problem of having useful logs. -Circolog is a syslog server which doesn't write to disk. Ever. It doesn't consume tons of RAM like elasticsearch/logstash, nor does it give you plots and a very long history. -It is however a very useful tool when you want to minimize disk writes. We wrote it thinking about user privacy: your logs can be just as sensitive as your database if you log too much! +Circolog is a syslog server which doesn't write to disk. Ever. It doesn't +consume tons of RAM like elasticsearch/logstash, nor does it give you plots and +a very long history. It is however a very useful tool when you want to minimize +disk writes. We wrote it thinking about user privacy: your logs can be just as +sensitive as your database if you log too much! -[Spiegone su casi d'uso, buttare i log, debuggare rapidamente magari applicazioni verbose, eccetera] +[Spiegone su casi d'uso, buttare i log, debuggare rapidamente magari +applicazioni verbose, eccetera] ## Why it is cool in 2 minutes -Keep your users safer +- It keeps your users safer. -Read logs conveniently: - filter with a proper (and easy) query language! +- Read logs conveniently: filter with a proper (and easy) query language! + `grep` is powerful, but sometimes you want something more expressive: - grep is powerful, but sometimes you want something more expressive: +``` +circolog-tail -where 'app_name=="apache" and message LIKE "%memory%"' +circolog-tail -where '(app_name"apache" OR app_name LIKE "php%") AND message LIKE "%memory%"' +circolog-tail -where 'app_name=="mysql" OR severity >= warning' +``` - circolog-tail -where 'app_name=="apache" and message LIKE "%memory%"' +- Colors: highlight severity and visually group related message - circolog-tail -where '(app_name"apache" OR app_name LIKE "php%") AND message LIKE "%memory%"' + [screenshot] - circolog-tail -where 'app_name=="mysql" OR severity >= warning' - - colors to highlight high severity and visually group related message - - [screenshot] - - It's not only about being nice, we swear! coloring logs also means reaching the most important entries easily, and grouping related entries together. - - - output logs in the format you prefer _now_ - - depending on what you're doing, log format might be useful... or distracting. For example, how many times have you used the `hostname` part of it? With the common disk-based logging, you need to choose once and for all how your logs will be saved. With circolog + It's not only about being nice, we swear! Coloring logs also means reaching the + most important entries easily, and grouping related entries together. Output + logs in the format you prefer _now_. Depending on what you're doing, log format + might be useful... or distracting. For example, how many times have you used + the `hostname` part of it? With the common disk-based logging, you need to + choose once and for all how your logs will be saved. -Hackable - - We think that circolog-tail is pretty cool, but you definitely can reuse simpler tools to get logs and filter them the way you prefer. clients can read logs using plain HTTP (or websocket). Most of the cool features of circolog-tail are actually implemented server-side, so you can use filters (or other options) wih any client. +- Hackable: we think that `circolog-tail` is pretty cool, but you definitely can + reuse simpler tools to get logs and filter them the way you prefer. Clients + can read logs using plain HTTP (or websocket). Most of the cool features of + `circolog-tail` are actually implemented server-side, so you can use filters + (or other options) with any client. -Fast, secure by default (?), easy deploy - - Those are features that you should expect, not be surprised of! circolog can easily process thousands of log entries per seconds, has sane defaults and can be deployed as a single binary +- Fast, secure by default (?), easy to deploy. Those are features that you + should expect, not be surprised of! Circolog can easily process thousands + of log entries per seconds, has sane defaults and can be deployed as a single + binary. ## Security considerations -While we try our best not to introduce vulnerabilities, this software is not meant to be exposed on the wider internet. -Beware of binding it on something different from localhost. +While we try our best not to introduce vulnerabilities, this software is not +meant to be exposed on the wider internet. Beware of binding it on something +different from `localhost`. -Even without being exposed, care must be given to socket permissions: don't let unprivileged users read your logs! -We suggest that you use a dedicated user/group to run circolog, and make root part of that group. +Even without being exposed, care must be given to socket permissions: don't let +unprivileged users read your logs! We suggest that you use a dedicated +user/group to run circolog, and make root part of that group. diff --git a/docs/docs/install.md b/docs/docs/install.md index 0e4349e..6d2f75a 100644 --- a/docs/docs/install.md +++ b/docs/docs/install.md @@ -1,25 +1,35 @@ ## Installation and Configuration -If you trust us, just get the compiled binaries and copy them to /usr/local/{s,}bin/ (but you shouldn't: [compiling -> altra pagina]). +If you trust us, just get the compiled binaries and copy them to +`/usr/local/{s,}bin/` (but you shouldn't: [TODO: compiling -> altra pagina]). -Running circologd is pretty easy: just run `circologd` by hand or in your favourite service manager: [here are some contrib scripts] for sysV, systemd, supervisor... but consider them as hints, don't just copy them! +Running circolog is pretty easy: just run `circologd` by hand or in your +favourite service manager: here are some contrib scripts for SysV, systemd, +supervisor... but consider them as hints, don't just copy them! + +[TODO: add scripts] ### System Integration -While circologd can be your only syslog daemon, you might want to couple it with another syslog to have the best of both worlds. For example, you could use rsyslog (or syslog-ng) to write important (ie: priority >= notice) log to persistent storage, letting circolog handle short-term but heavily detailed logs. +While circologd can be your only syslog daemon, you might want to couple it +with another syslog to have the best of both worlds. For example, you could use +rsyslog (or syslog-ng) to write important (ie: `priority >= notice`) log to +persistent storage, letting circolog handle short-term but heavily detailed +logs. -Or you could make circolog get messages from journald. While possibilities are endless, we tried to document some common setups. +Or you could make circolog get messages from `journald`. While possibilities are +endless, we tried to document some common setups. #### Use Case: circologd as the only syslog -[mica lo so se funziona, credo di no, ma proviamo] -bind it to /dev/log +[mica lo so se funziona, credo di no, ma proviamo] bind it to `/dev/log`. #### Use Case: circologd receiving messages from syslog-ng #### Use Case: circologd receiving messages from journald -[TODO: il codice va ancora scritto: bisogna bindare /run/systemd/journal/syslog con formato != rfc5424 +[TODO: il codice va ancora scritto: bisogna bindare /run/systemd/journal/syslog +con formato != rfc5424] diff --git a/docs/docs/query.md b/docs/docs/query.md index 0ad1173..99048bd 100644 --- a/docs/docs/query.md +++ b/docs/docs/query.md @@ -1,14 +1,15 @@ Query language -=================== +============== -circolog uses a sql-inspired query language. If you know SQL, then you can use "where clauses" in circolog. If -you don't know SQL, don't worry: the language is easy enough for you to learn the most basic queries without -worrying too much. +Circolog uses a sql-inspired query language. If you know SQL, then you can use +"where clauses" in circolog. If you don't know SQL, don't worry: the language +is easy enough for you to learn the most basic queries without worrying too +much. You can only filter the rows, you can't sort them or group them in any way. Reference ------------ +--------- Available fields: - `message`: the string with the main information @@ -19,3 +20,7 @@ Available fields: - `severity`: an integer describing severity +Examples +-------- + +TODO