Cacti - Using Data Query

Cacti's documentation is actually pretty comprehensive, and I would strongly recommend it.  I still write this only because I feel some parts may not be detailed enough in the doc.
Here is the URL for "data queries"
http://docs.cacti.net/manual:100:3a_advanced_topics.3_data_queries#data_queries

Data Query is a very useful and powerful tool to create graphs. There are basically two ways to do it in Cacti - (1) With SNMP, (2) With a script

With an external script, you can virtually do anything with it to accomplish some very complicated tasks.  But whenever it is possible to be done in SNMP query, please do it.  Running spine with pure snmp calls is often much more efficient than calling external scripts.

SNMP Query

(1) XML Template File:

• Define an index
  <oid_index>, <oid_index_parse>, <oid_suffix>
• Define input parameters
• Define output parameters

IF-MIB example:
This is by far the simplest type and is bundled with Cacti in "resource/snmp_queries.interface.xml".  First, it has a OID that describes all its indexes
<oid_index>.1.3.6.1.2.1.2.2.1.1</oid_index>
e.g. in my VM, snmpwalk -v2c -c xxxxxx localhost .1.3.6.1.2.1.2.2.1.1
IF-MIB::ifIndex.1 = INTEGER: 1
IF-MIB::ifIndex.2 = INTEGER: 2

And then we would use the index obtained from <oid_index> to reference other OIDs

e.g. Description

snmpwalk -v2c -c xxxxx localhost .1.3.6.1.2.1.2.2.1.2

IF-MIB::ifDescr.1 = STRING: lo

IF-MIB::ifDescr.2 = STRING: enp0s3

e.g. OutOctets

snmpwalk -v2c -c xxxxx localhost .1.3.6.1.2.1.2.2.1.16
IF-MIB::ifOutOctets.1 = Counter32: 2792348  <= this is "lo" out octets
IF-MIB::ifOutOctets.2 = Counter32: 8936144  <= this is "enp0s3" out octets


However, reality is cruel and the MIB world isn't always so nice...  what if the index OID doesn't exist?  Take IF-MIB as an example, let's pretend that "<oid_index>.1.3.6.1.2.1.2.2.1.1</oid_index>" isn't available, what can we do?  If you look at the "Description" OID again, you may have noticed that it already contains the "index" as part of its OID!
snmpwalk -v2c -c xxxxx localhost .1.3.6.1.2.1.2.2.1.2
IF-MIB::ifDescr.1 = STRING: lo
IF-MIB::ifDescr.2 = STRING: enp0s3

Let's convert it to raw format to have a clearer look (-On)
snmpwalk -v2c -c public -On localhost .1.3.6.1.2.1.2.2.1.2
.1.3.6.1.2.1.2.2.1.2.1 = STRING: lo
.1.3.6.1.2.1.2.2.1.2.2 = STRING: enp0s3

Here comes the "<oid_index_parse>" parameter, we want to extract part of the OID as the index. One thing though, to use this parameter you should at least have some basic knowledge on regular expression.
e.g.
<oid_index>.1.3.6.1.2.1.2.2.1.2</oid_index>
<oid_index_parse>OID/REGEXP:.*\.([0-9]{1,})$</oid_index_parse>

Now, in interface.xml, if you replace:
        <oid_index>.1.3.6.1.2.1.2.2.1.1</oid_index>
        <oid_num_indexes>.1.3.6.1.2.1.2.1.0</oid_num_indexes>

With:

        <oid_index>.1.3.6.1.2.1.2.2.1.2</oid_index>
        <oid_index_parse>OID/REGEXP:.*\.([0-9]{1,})$</oid_index_parse>

It should just work fine with the same behavior.

Below is some further examples from Cacti Doc, strongly recommended to go through it.
http://docs.cacti.net/howto:data_query_templates

After you are done with the index part, the rest is relatively simple. Input are parameters that would help you create and describe your graph, while output are values that would actually appear in your graphs. Read through interface.xml, and create some graphs then you would fully understand.  To do it, just create a new device in "Management->devices", or simply use "Local Linux Machine" as a starting point.  Go to "Associated Data Queries" at the bottom, select "SNMP - Interface Statistics" and click "Add".  Then click "Create Graphs for this Device" near the top right corner.

One more note on the output parameter, some OIDs may not simply append the "index" part to the end... e.g. 1.2.3.4.5.6.index.1
<Sample> 
<name>Sample1</name> 
<method>get</method> 
<source>value</source> 
<direction>output</direction> 
<oid>.1.2.3.4.5.6</oid> 
<oid_suffix>1</oid_suffix> </Sample> 


Script Data Query

The idea is the same, but we do it in a script rather than the built-in snmp engine.  Take "resource/script_server/host_cpu.xml" and "scripts/ss_host_cpu.php" as example.

script to run
<script_path>|path_cacti|/scripts/ss_host_cpu.php</script_path>

function name
<script_function>ss_host_cpu</script_function>

arguments
<arg_prepend>|host_hostname| |host_id| |host_snmp_version|:|host_snmp_port|:|host_snmp_timeout|:|host_ping_retries|:|host_max_oids|:|host_snmp_community|:|host_snmp_username|:|host_snmp_password|:|host_snmp_auth_protocol|:|host_snmp_priv_passphrase|:|host_snmp_priv_protocol|:|host_snmp_context|</arg_prepend>

These refer to "$cmd" in the script
<arg_index>index</arg_index>
<arg_query>query</arg_query>
<arg_get>get</arg_get>
<arg_num_indexes>num_indexes</arg_num_indexes>

The <query_name> paramters refer to "$arg1" in the script
<hrProcessorFrwID>
    <name>Processor Index Number</name>
    <direction>input</direction>
    <query_name>index</query_name>
</hrProcessorFrwID>
<hrProcessorLoad>
    <name>Processor Usage</name>
    <direction>output</direction>
    <query_name>usage</query_name>
</hrProcessorLoad>

Now we look at the "scripts/ss_host_cpu.php", you will find it has a funciton:
function ss_host_cpu($hostname, $host_id, $snmp_auth, $cmd, $arg1 = '', $arg2 = '')

Let's take a look at the argument part, $hostname refers to |hostname|, $hostid to |host_id|, $snmp_auth to the long colon separated snmp parameters. $cmd is the action, $arg1 is the "query_name" and $arg2 is the index obtained from "index" cmd.

Test by calling from command line:
# php -q scripts/ss_host_cpu.php 127.0.0.1 1 1:161:500:1:5:"my_community":"":"":"":"":"":"" index
0
1
2

3

# php -q scripts/ss_host_cpu.php 127.0.0.1 1 1:161:500:1:5:"my_community":"":"":"":"":"":"" num_indexes
4

Query CPU indexes
# php -q scripts/ss_host_cpu.php 127.0.0.1 1 1:161:500:1:5:"my_community":"":"":"":"":"":"" query index
0!0
1!1
2!2
3!3

Query CPU usage
# php -q scripts/ss_host_cpu.php 127.0.0.1 1 1:161:500:1:5:"my_community":"":"":"":"":"":"" query usage
0!1
1!1
2!1
3!1

Well, does it look right?  Let's make the CPU busy and try again.
run a busy loop by
# while [ 1 ]; do echo 123 > /dev/null; done

# php -q scripts/ss_host_cpu.php 127.0.0.1 1 1:161:500:1:5:"my_community":"":"":"":"":"":"" query usage

0!1

1!1
2!41
3!1

Get individual

# php -q scripts/ss_host_cpu.php 127.0.0.1 1 1:161:500:1:5:"my_community":"":"":"":"":"":"" get usage 0

1

Once again, do read the official doc if you have time.
http://docs.cacti.net/manual:100:3a_advanced_topics.3d_script_data_query_walkthrough#script_data_query_walkthrough

After you're done with the xml file, go to "Data Queries" and create a new one that points to your xml file.

Unfortunately, we are still few steps away from creating a graph with "Data Query", but the rest parts are pretty straight forward. The easiest way to understand it is to go through the network interface examples again, or read through Cacti's official doc.  I will leave out the details as I am lazy...
http://docs.cacti.net/manual:100:3a_advanced_topics.3_data_queries#data_queries

(2) Data Source Template

• Interface - Errors/Discards
• Interface - Non-Unicast Packets
• Interface - Traffic
• Interface - Unicast Packets

(3) Graph Template

• Interface - Errors/Discards
• Interface - Non-Unicast Packets
• Interface - Traffic (bits/sec, 95th Percentile)
• Interface - Traffic (bits/sec, Total Bandwidth)
• Interface - Traffic (bits/sec)
• Interface - Traffic (bytes/sec, Total Bandwidth)
• Interface - Traffic (bytes/sec)
• Interface - Unicast Packets

DEBUG: Use the "Verbose Query" feature in Device, it will help a lot when you are stuck.