RRDTOOL(1) rrdtool RRDTOOL(1)
NAME
rrdtool - Round Robin Database Tool
SYNOPSIS
rrdtool - [workdir] function
DESCRIPTION
OVERVIEW
It is pretty easy to gather status information from all sorts of
things, ranging from the temperature in your office to the number of
octets which have passed through the FDDI interface of your router. But
it is not so trivial to store this data in an efficient and systematic
manner. This is where RRDtool comes in handy. It lets you log and ana
lyze the data you gather from all kinds of data-sources (DS). The data
analysis part of RRDtool is based on the ability to quickly generate
graphical representations of the data values collected over a definable
time period.
In this man page you will find general information on the design and
functionality of the Round Robin Database Tool (RRDtool). For a more
detailed description of how to use the individual functions of RRDtool
check the corresponding man page.
For an introduction to the usage of RRDtool make sure you consult the
rrdtutorial.
FUNCTIONS
While the man pages talk of command line switches you have to set in
order to make RRDtool work it is important to note that RRDtool can be
remotely controlled through a set of pipes. This saves a considerable
amount of startup time when you plan to make RRDtool do a lot of things
quickly. Check the section on "Remote Control" further down. There is
also a number of language bindings for RRDtool which allow you to use
it directly from perl, python, tcl, php, etc.
create Set up a new Round Robin Database (RRD). Check rrdcreate.
update Store new data values into an RRD. Check rrdupdate.
updatev Operationally equivalent to update except for output. Check
rrdupdate.
graph Create a graph from data stored in one or several RRDs. Apart
from generating graphs, data can also be extracted to stdout.
Check rrdgraph.
dump Dump the contents of an RRD in plain ASCII. In connection with
restore you can use this to move an RRD from one computer
architecture to another. Check rrddump.
restore Restore an RRD in XML format to a binary RRD. Check rrdrestore
fetch Get data for a certain time period from a RRD. The graph func
tion uses fetch to retrieve its data from an RRD. Check rrd
fetch.
tune Alter setup of an RRD. Check rrdtune.
last Find the last update time of an RRD. Check rrdlast.
info Get information about an RRD. Check rrdinfo.
rrdresize
Change the size of individual RRAs. This is dangerous! Check
rrdresize.
xport Export data retrieved from one or several RRDs. Check rrdxport
rrdcgi This is a standalone tool for producing RRD graphs on the fly.
Check rrdcgi.
HOW DOES RRDTOOL WORK?
Data Acquisition
When monitoring the state of a system, it is convenient to have
the data available at a constant time interval. Unfortunately,
you may not always be able to fetch data at exactly the time
you want to. Therefore RRDtool lets you update the logfile at
any time you want. It will automatically interpolate the value
of the data-source (DS) at the latest official time-slot
(intervall) and write this interpolated value to the log. The
original value you have supplied is stored as well and is also
taken into account when interpolating the next log entry.
Consolidation
You may log data at a 1 minute interval, but you might also be
interested to know the development of the data over the last
year. You could do this by simply storing the data in 1 minute
intervals for the whole year. While this would take consider
able disk space it would also take a lot of time to analyze the
data when you wanted to create a graph covering the whole year.
RRDtool offers a solution to this problem through its data con
solidation feature. When setting up an Round Robin Database
(RRD), you can define at which interval this consolidation
should occur, and what consolidation function (CF) (average,
minimum, maximum, total, last) should be used to build the con
solidated values (see rrdcreate). You can define any number of
different consolidation setups within one RRD. They will all be
maintained on the fly when new data is loaded into the RRD.
Round Robin Archives
Data values of the same consolidation setup are stored into
Round Robin Archives (RRA). This is a very efficient manner to
store data for a certain amount of time, while using a known
and constant amount of storage space.
It works like this: If you want to store 1000 values in 5
minute interval, RRDtool will allocate space for 1000 data
values and a header area. In the header it will store a pointer
telling which slots (value) in the storage area was last writ
ten to. New values are written to the Round Robin Archive in,
you guessed it, a round robin manner. This automatically limits
the history to the last 1000 values (in our example). Because
you can define several RRAs within a single RRD, you can setup
another one, for storing 750 data values at a 2 hour interval,
for example, and thus keep a log for the last two months at a
lower resolution.
The use of RRAs guarantees that the RRD does not grow over time
and that old data is automatically eliminated. By using the
consolidation feature, you can still keep data for a very long
time, while gradually reducing the resolution of the data along
the time axis.
Using different consolidation functions (CF) allows you to
store exactly the type of information that actually interests
you: the maximum one minute traffic on the LAN, the minimum
temperature of your wine cellar, the total minutes of down
time, etc.
Unknown Data
As mentioned earlier, the RRD stores data at a constant inter
val. Sometimes it may happen that no new data is available when
a value has to be written to the RRD. Data acquisition may not
be possible for one reason or other. With RRDtool you can han
dle these situations by storing an *UNKNOWN* value into the
database. The value *UNKNOWN* is supported through all the
functions of the tool. When consolidating a data set, the
amount of *UNKNOWN* data values is accounted for and when a new
consolidated value is ready to be written to its Round Robin
Archive (RRA), a validity check is performed to make sure that
the percentage of unknown values in the data point is above a
configurable level. If not, an *UNKNOWN* value will be written
to the RRA.
Graphing
RRDtool allows you to generate reports in numerical and graphi
cal form based on the data stored in one or several RRDs. The
graphing feature is fully configurable. Size, color and con
tents of the graph can be defined freely. Check rrdgraph for
more information on this.
Aberrant Behavior Detection
by Jake Brutlag
RRDtool provides the building blocks for near real-time aber
rant behavior detection. These components include:
* An algorithm for predicting the value of a time series one
time step into the future.
* A measure of deviation between predicted and observed val
ues.
* A mechanism to decide if and when an observed value or
sequence of observed values is too deviant from the pre
dicted value(s).
Here is a brief explanation of these components:
The Holt-Winters time series forecasting algorithm is an on-
line (or incremental) algorithm that adaptively predicts future
observations in a time series. Its forecast is the sum of three
components: a baseline (or intercept), a linear trend over time
(or slope), and a seasonal coefficient (a periodic effect, such
as a daily cycle). There is one seasonal coefficient for each
time point in the period (cycle). After a value is observed,
each of these components is updated via exponential smoothing.
This means that the algorithm "learns" from past values and
uses them to predict the future. The rate of adaptation is gov
erned by 3 parameters, alpha (intercept), beta (slope), and
gamma (seasonal). The prediction can also be viewed as a
smoothed value for the time series.
The measure of deviation is a seasonal weighted absolute devia
tion. The term seasonal means deviation is measured separately
for each time point in the seasonal cycle. As with Holt-Winters
forecasting, deviation is predicted using the measure computed
from past values (but only at that point in the seasonal
cycle). After the value is observed, the algorithm learns from
the observed value via exponential smoothing. Confidence bands
for the observed time series are generated by scaling the
sequence of predicted deviation values (we usually think of the
sequence as a continuous line rather than a set of discrete
points).
Aberrant behavior (a potential failure) is reported whenever
the number of times the observed value violates the confidence
bands meets or exceeds a specified threshold within a specified
temporal window (e.g. 5 violations during the past 45 minutes
with a value observed every 5 minutes).
This functionality is embedded in a set of related RRAs. In
particular, a FAILURES RRA logs potential failures. With these
data you could, for example, use a front-end application to
RRDtool to initiate real-time alerts.
For a detailed description on how to set this up, see rrdcre
ate.
REMOTE CONTROL
When you start RRDtool with the command line option - it waits for
input via standard input (STDIN). With this feature you can improve
performance by attaching RRDtool to another process (MRTG is one exam
ple) through a set of pipes. Over these pipes RRDtool accepts the same
arguments as on the command line and some special commands like quit,
cd, mkdir and ls. For detailed help on the server commands type:
rrdtool help cdmkdirpwdlsquit
When a command is completed, RRDtool will print the string "OK",
followed by timing information of the form u:usertime s:systemtime.
Both values are the running totals of seconds since RRDtool was
started. If an error occurs, a line of the form "ERROR:" Description
of error will be printed instead. RRDtool will not abort, unless some
thing realy serious happens. If a workdir is specified and the UID is
0, RRDtool will do a chroot to that workdir. If the UID is not 0, RRD
tool only changes the current directory to workdir.
RRD Server
If you want to create a RRD-Server, you must choose a TCP/IP Service
number and add them to /etc/services like this:
rrdsrv 13900/tcp # RRD server
Attention: the TCP port 13900 isnt officially registered for rrdsrv.
You can use any unused port in your services file, but the server and
the client system must use the same port, of course.
With this configuration you can add RRDtool as meta-server to
/etc/inetd.conf. For example:
rrdsrv stream tcp nowait root /opt/rrd/bin/rrdtool rrdtool - /var/rrd
Dont forget to create the database directory /var/rrd and reinitialize
your inetd.
If all was setup correctly, you can access the server with perl sock
ets, tools like netcat, or in a quick interactive test by using telnet
localhost rrdsrv.
NOTE: that there is no authentication with this feature! Do not setup
such a port unless you are sure what you are doing.
SEE ALSO
rrdcreate, rrdupdate, rrdgraph, rrddump, rrdfetch, rrdtune, rrdlast,
rrdxport
BUGS
Bugs? Features!
AUTHOR
Tobias Oetiker
1.2.15 2006-07-14 RRDTOOL(1)
|
