- English
- 日本語
Abstract
Hello everybody, It’s me candle.
In this time, let’s get the rails 4 production.log by td-agent.
I have been written same article, but there was lack of explanation and etc, I will rewrite it again.
Also, I will show you how to get Rails 5 production.log at a later date.
Precondition
You must be able to execute ruby on rails in production mode.
It is ok in any environment as long as it has executable. Such as webrick, nginx, puma and apache.
td-agent or fluentd is instlled.
It is assumed that td-agent or fluentd is installed.
You may change the rails log format.
I think that the format of the production log of rails is the next image by default.
You can change the this log format easily.
The below is a code without any change from the default format.
class Logger::Formatter def call(severity, time, progname, msg) format = "%s, [%s #%d] %5s -- %s: %sn" format % [severity[0], "#{time.strftime('%Y-%m-%dT%H:%M:%S')}.#{'%06d' % time.usec.to_s}", $$, severity, progname, msg2str(msg)] end end
There is information that is not necessary for rails production log.
It is “Render partial” information.
The below text is sample.
I, [2017-03-24T06:19:15.315840 #5689] INFO — : Rendered users/registrations/new.html.haml within layouts/application (6.6ms)
I, [2017-03-24T06:19:15.318778 #5689] INFO — : Rendered layouts/_header.html.haml (0.6ms)
I, [2017-03-24T06:19:15.320581 #5689] INFO — : Rendered layouts/_footer.html.haml (0.8ms)
I think that this information increase the log file size
and this information is not necessary in the production mode.
It is ok if you don’t change that or not.
Open the config/environment.rb
Change the format like this.
If you are using erb please change “.haml” part to “.erb”.
class Logger::Formatter def call(severity, time, progname, msg) if !msg.include?('.haml') format = "%s, [%s #%d] %5s -- %s: %sn" format % [severity[0], "#{time.strftime('%Y-%m-%dT%H:%M:%S')}.#{'%06d' % time.usec.to_s}", $$, severity, progname, msg2str(msg)] end end end
Setting of rails is over.
Preparing to get rails log with td-agent
Alright, let’s prepare for td-agent side.
My rails and td-agent environment is like this.
Td-agent is running with CentOS 6.5.
Rails application is running on the unicorn + nginx.
Please prepare those servers environment.
Set up td-agent
The biggest problem with rails 4 production log is that Fatal errors are displayed over multiple lines.
Fluentd gets a log each one line.
The multiple line errors are collected as separate logs.
NoMethodError (undefined method `each’ for nil:NilClass):
app/controllers/pages_controller.rb:7:in `about’
So, we use “format multiline”.
This means that the part not matching the regular expression written with “format_firstline” is regarded as the same as the matched part even if it is new line, and it is stored in one log.
Open the /etc/td-agent/td-agent.conf and write it.
<source> @type tail path /usr/share/nginx/alice4/log/production.log tag rails.production pos_file /var/log/td-agent/rails.production.pos format multiline format_firstline /^.,/ format1 /^(?<log_initial>.*), [(?<date>d{1,4}-d{1,2}-d{1,2}Td{1,2}:d{1,2}:d{1,2}.d{1,6}) (?<process_id>[^ ]*)] *(?<log_level>[^ ]*) -- : (?<message>[^']*[^]]*)/ </source> <match rails.production> @type stdout </match>
With the “path” reference the production.log in the log folder of the rails project.
Please rewrite the reference path to your environment.
“tag rails.production” tagged a collected log.
With the “pos_file /var/log/td-agent/rails.production.pos” specifies the location of temporary files.
In the “format” declares “multiline”. It is possible to collect logs of multiple lines.
the meaning of “format_firstline / ^., /” regular expression is “1 character +,” at the beginning of the line.
This corresponds to the “I,” part of the rails production log.
“format1” matches the format of rails production.log.
In the “<match rails.production>” gets the tagged log and output it.
This is the setting for td-agent to get rails log.
Getting log and check it
Restart the td-agent.
sudo serice td-agent restart
Lauch the rails application in production.
Let’s access.
The td-agent log is output to the following location.
/var/log/td-agent/td-agent.log
Check with the tail command.
sudo tail -f /var/log/td-agent/td-agent.log
The most important thing is the part of “FATAL”.
The messages are gathered to one.
Conclusion
We tried to collect Rails 4 production.log.