boyska/circolog

Fork 1

receive messages from journald #23

New issue

Closed

opened 2019-04-26 10:46:44 +02:00 by boyska · 5 comments

boyska commented

2019-04-26 10:46:44 +02:00

Owner

Many distros use systemd-journald as main syslog. Receiving messages from it is a valuable thing to do.

To do that, a user just needs to bind to /run/systemd/journal/syslog. The problem is a misinterpretation of the message formats, leading to wrong parsing.

Many distros use systemd-journald as main syslog. Receiving messages from it is a valuable thing to do. To do that, a user just needs to bind to `/run/systemd/journal/syslog`. The problem is a misinterpretation of the message formats, leading to wrong parsing.

boyska commented

2019-04-26 10:50:47 +02:00

Author

Owner

See https://git.lattuga.net/boyska/circolog/src/master/cmd/circologd/main.go#L46

instead of syslog.Automatic, user should be able to force a certain input format

(or is it "just" a bug in syslog.Automatic? perhaps it's worth reporting to gopkg.in/mcuadros/go-syslog.v2 maintainers)

See https://git.lattuga.net/boyska/circolog/src/master/cmd/circologd/main.go#L46 instead of `syslog.Automatic`, user should be able to force a certain input format (or is it "just" a bug in syslog.Automatic? perhaps it's worth reporting to gopkg.in/mcuadros/go-syslog.v2 maintainers)

blallo commented

2019-05-01 15:58:13 +02:00

Collaborator

I prepared a branch with an ad-hoc commit. Still not tested (and no test written).

https://git.lattuga.net/boyska/circolog/src/set-syslog-fmt

I prepared a branch with an ad-hoc commit. Still not tested (and no test written). https://git.lattuga.net/boyska/circolog/src/set-syslog-fmt

boyska commented

2019-05-01 20:35:57 +02:00

Author

Owner

here are some random notes to test inside a clean debian container:

sudo cp $GOPATH/bin/circolog* /var/lib/machines/stretch-rsyslog/usr/local/sbin && sudo systemd-nspawn -xbD /var/lib/machines/stretch-rsyslog

systemctl stop rsyslog syslog.socket; rm /run/systemd/journal/syslog ; circologd -syslogd-socket /run/systemd/journal/syslog -log-fmt rfc3164 &

here are some random notes to test inside a clean debian container: ``` sudo cp $GOPATH/bin/circolog* /var/lib/machines/stretch-rsyslog/usr/local/sbin && sudo systemd-nspawn -xbD /var/lib/machines/stretch-rsyslog ``` ``` systemctl stop rsyslog syslog.socket; rm /run/systemd/journal/syslog ; circologd -syslogd-socket /run/systemd/journal/syslog -log-fmt rfc3164 & ```

boyska commented

2019-05-01 20:54:44 +02:00

Author

Owner

some more material

here is a line generated by journald

<78>May  1 20:51:02 cron[153]: (CRON) INFO (Skipping @reboot jobs -- not system startup)

and here is one generated by logger --rfc3164 (as provided by util-linux)

<13>May  1 20:51:40 myhostname root: ciao

I think the only difference is the presence of the hostname. So I wonder if that's really required by rfc3164, and whether the bug is therefore in go-syslog.v2 or in journald

some more material here is a line generated by journald ``` <78>May 1 20:51:02 cron[153]: (CRON) INFO (Skipping @reboot jobs -- not system startup) ``` and here is one generated by `logger --rfc3164` (as provided by util-linux) ``` <13>May 1 20:51:40 myhostname root: ciao ``` I think the only difference is the presence of the hostname. So I wonder if that's really required by rfc3164, and whether the bug is therefore in `go-syslog.v2` or in `journald`

boyska referenced this issue from a commit

2019-05-02 12:33:27 +02:00

Merge remote-tracking branch 'blallo/set-syslog-fmt'...

blallo commented

2019-05-02 14:28:26 +02:00

Collaborator

I can confirm that now the messages received from journald are correctly parsed. In a debian stable vm, I did:

systemctl stop rsyslog syslog.socket; rm /run/systemd/journal/syslog

Then I started circologd, forcing the format:

~/circolog $ ./circologd -syslogd-socket /run/systemd/journal/syslog -log-fmt rfc3164

Listening with circolog-tail I got:

# Sent from another terminal
$ logger "ciao"
$ logger --rfc3164  "ciao"                                                                                  
$ logger --rfc5424  "ciao"

# Output from circolog-tail
$ ./circolog-tail
2019/05/02 14:18:56 connecting to localhost:9080
2019/05/02 14:18:56 connected! ws://localhost:9080/ws?fmt=bson
02 May 19 16:19 CEST testvm root: notice ciao
02 May 19 16:19 CEST testvm root: notice ciao
02 May 19 14:20 CEST /run/systemd/journal/dev-log : notice 1 2019-05-02T14:20:00.278040+02:00 testvm root - - [timeQuality tzKnown="1" isSynced="1" syncAccuracy="85500"] ciao

Restarting circologd with log-fmt set to auto:

~/circolog $ ./circologd -syslogd-socket /run/systemd/journal/syslog -log-fmt auto

The same experiment above works smoothly:

# From another terminal
$ logger --rfc5424  "ciao"
$ logger --rfc3164  "ciao"                                                                                  
$ logger "ciao"

# Output from circolog-tail
$ ./circolog-tail
2019/05/02 14:20:25 connecting to localhost:9080
2019/05/02 14:20:25 connected! ws://localhost:9080/ws?fmt=bson
02 May 19 14:20 CEST testvm root: notice ciao
02 May 19 16:20 CEST testvm root: notice ciao
02 May 19 16:20 CEST testvm root: notice ciao

I can confirm that now the messages received from `journald` are correctly parsed. In a debian stable vm, I did: ``` systemctl stop rsyslog syslog.socket; rm /run/systemd/journal/syslog ``` Then I started `circologd`, forcing the format: ``` ~/circolog $ ./circologd -syslogd-socket /run/systemd/journal/syslog -log-fmt rfc3164 ``` Listening with `circolog-tail` I got: ``` # Sent from another terminal $ logger "ciao" $ logger --rfc3164 "ciao" $ logger --rfc5424 "ciao" # Output from circolog-tail $ ./circolog-tail 2019/05/02 14:18:56 connecting to localhost:9080 2019/05/02 14:18:56 connected! ws://localhost:9080/ws?fmt=bson 02 May 19 16:19 CEST testvm root: notice ciao 02 May 19 16:19 CEST testvm root: notice ciao 02 May 19 14:20 CEST /run/systemd/journal/dev-log : notice 1 2019-05-02T14:20:00.278040+02:00 testvm root - - [timeQuality tzKnown="1" isSynced="1" syncAccuracy="85500"] ciao ``` Restarting `circologd` with `log-fmt` set to auto: ``` ~/circolog $ ./circologd -syslogd-socket /run/systemd/journal/syslog -log-fmt auto ``` The same experiment above works smoothly: ``` # From another terminal $ logger --rfc5424 "ciao" $ logger --rfc3164 "ciao" $ logger "ciao" # Output from circolog-tail $ ./circolog-tail 2019/05/02 14:20:25 connecting to localhost:9080 2019/05/02 14:20:25 connected! ws://localhost:9080/ws?fmt=bson 02 May 19 14:20 CEST testvm root: notice ciao 02 May 19 16:20 CEST testvm root: notice ciao 02 May 19 16:20 CEST testvm root: notice ciao ```

boyska closed this issue

2019-05-02 19:05:59 +02:00