Agenda: exercises-smokeping.html

File exercises-smokeping.html, 11.7 KB (added by admin, 8 years ago)
Line 
1<html>
2<head>
3<title>Smokeping Exercises</title>
4</head>
5<body>
6<pre>
7
8Network Monitoring and Management
9
10Smokeping
11---------
12
13Notes:
14------
15* Commands preceded with "$" imply that you should execute the command as
16  a general user - not as root.
17* Commands preceded with "#" imply that you should be working as root.
18* Commands with more specific command lines (e.g. "RTR-GW>" or "mysql>")
19  imply that you are executing commands on remote equipment, or within
20  another program.
21
22Exercises
23----------
24
250. Log in to your PC or open a terminal window as the sysadmn user.
26
27Once you are logged in you can continue with these exercises.
28
291. Install Smokeping
30
31    $ sudo apt-get install smokeping
32
332. Initial Configuration
34
35    $ cd /etc/smokeping/config.d
36    $ ls -l
37
38    -rwxr-xr-x 1 root root  578 2010-02-26 01:55 Alerts
39    -rwxr-xr-x 1 root root  237 2010-02-26 01:55 Database
40    -rwxr-xr-x 1 root root  413 2010-02-26 05:40 General
41    -rwxr-xr-x 1 root root  271 2010-02-26 01:55 pathnames
42    -rwxr-xr-x 1 root root  859 2010-02-26 01:55 Presentation
43    -rwxr-xr-x 1 root root  116 2010-02-26 01:55 Probes
44    -rwxr-xr-x 1 root root  155 2010-02-26 01:55 Slaves
45    -rwxr-xr-x 1 root root 8990 2010-02-26 06:30 Targets
46 
47    $ sudo vi General
48
49        Change the following lines:
50
51        owner    = NOC
52        contact  = sysadm@localhost
53        cgiurl   = http://localhost/cgi-bin/smokeping.cgi
54        mailhost = localhost
55
56        Save the file and exit. Now let's restart the
57        Smokeping service to verify that no mistakes have been made
58        before going any further:
59
60        $ sudo service smokeping restart
61
622. Configure monitoring of devices
63
64        The majority of your time and work configuring Smokeping
65        will be done in the file /etc/smokeping/config.d/Targets.
66       
67        For this class please do the following:
68
69        Use the default FPing probe to check:
70
71      - all the student PCs
72      - classroom NOC
73      - switches
74      - routers
75     
76    You can use the classroom Network Diagram on the classroom wiki to
77    figure out addresses for each item, etc.
78
79    ###############################################################################
80    Please note - a complete set of configuration files is available on the class
81    web server at http://noc.ws.nsrc.org/configs
82   
83    We strongly suggest you review these files to see what a complete set of
84    Smokeping configuration files looks like. To review the Targets file the direct
85    link is:
86   
87        <a href="http://noc/configs/etc/smokeping/config.d/Targets">http://noc/configs/etc/smokeping/config.d/Targets</a> 
88   
89    ###############################################################################
90   
91    Create some hierarchy to the Smokeping menu for your
92    checks. Such as:
93
94        +Local
95
96        menu = Network Monitoring and Management
97        title = NOC Server for Network Monitoring Class
98
99        ++LocalMachine
100
101        menu = The NOC@NetManage
102        title = The NOC@NetManage
103        host = localhost
104
105    #
106    # Classroom PCs
107    #
108
109        ++PCs
110
111    +++pc1
112    menu = pc1
113    title = pc1
114    host = pc1
115
116    +++pc2
117    menu = pc2
118    title = pc2
119    host = pc2
120       
121    Save the file and restart Smokeping:
122
123        $ sudo service smokeping restart
124
125    Go to your browser and check the Smokeping page:
126
127        http://pcN.ws.nsrc.org/cgi-bin/smokeping.cgi
128
129    If everything is looking OK, continue adding:
130
131        ++Routers
132
133        +++rtr
134        menu = rtr
135        title = Gateway Router
136        host = rtr
137
138        +++rtr1
139        menu = rtr1
140        title = Router 1, Group 1
141        host = rtr1
142
143    ++Switch
144
145    ++sw
146    menu = sw
147    title = Backbone Switch
148    host = sw
149   
150        ...
151
152    Save the file, restart smokeping, and check
153    your browser again.
154
1553. Add new probes
156
157    The current entry in Probes is fine, but if you wish to
158    use additional Smokeping checks you can add them in here
159    and you can specify their default behavior. You can do
160    this, as well, in the Targets file if you wish.
161
162    Here is an example of a Probes file that would specify
163    what to use to check for HTTP and DNS latency as well as
164    the FPing probe that is used for ping latency:
165
166       $ sudo vi Probes
167
168        *** Probes ***
169
170        + FPing
171
172        binary = /usr/bin/fping
173
174        + EchoPingHttp
175
176        + DNS
177        binary = /usr/bin/dig
178        pings = 5
179        step = 180
180        lookup = www.nsrc.org
181
182        Save the file.
183
1844. Add HTTP latency checks
185
186    Now edit your Targets again:
187
188        $ sudo vi Targets
189       
190    Add a check for HTTP latency for all the classroom PCs.
191    This will mean adding another category, such as:
192
193    ++HTTP Servers
194        probe = EchoPingHttp
195        menu = HTTP Response
196    title = HTTP Response Student PCs
197
198        +++pc1
199        menu = pc1
200        title = PC1 HTTP Response Time
201        host = pc1
202
203        +++pc1
204        host = pc2
205        title = PC2 HTTP Response Time
206        host = pc2
207       
208        ...
209
210    If you have time, consider checking some machines that are
211    external to our classroom and the conference (your organization's
212    website, a popular web page, etc...)
213
2145. Add DNS Latency Checks
215
216    You can check either or both internal or external names using
217    the DNS latency probe.
218
219    Add a menu hierarchy for DNS Latency. Check an external address
220    (nsrc.org) and an internal address (noc). This will look something
221    like this (in Targets):
222
223        +DNS
224        probe = DNS
225        menu = External DNS Check
226        title = DNS Latency
227
228        ++nsrc
229        host = nsrc.org
230
231        ++noc
232        host = noc.mgmt
233
234    Exit and save your changes to the file Targets.
235
236    Restart Smokeping to see the changes:
237
238    $ sudo service smokeping restart
239
240    Look at additional Smokeping probes and consider implementing
241    some of them:
242
243        http://oss.oetiker.ch/smokeping/probe/index.en.html
244
245    As trying to explain all syntactical details of how the file
246    /etc/smokeping/config.d/Targets is used would require several
247    pages we will go through some examples in class, and you can
248    refer to the Smokeping configuration files that are in use on
249    the classroom NOC box by going to:
250
251        http://noc/configs/etc/smokeping
252        http://noc/configs/etc/smokeping/config.d
253
254
2556. Send Smokeping alerts
256
257    $ sudo vi Alerts
258
259    Update the top of the file where it says:
260
261        *** Alerts ***
262        to = alertee@address.somewhere
263        from = smokealert@company.xy
264
265    to include a proper "to" and "from" field for your server.
266    Something like:
267
268        *** Alerts ***
269        to = sysadm@localhost
270        from = smokeping-alert@localhost
271
272    If you have installed RT, you can instead send your alerts
273    to an existing RT queue:
274
275        *** Alerts ***
276        to = net@localhost
277
278    At the end of the file, add another alert like this:
279
280    +anydelay
281    type = rtt
282    # in milliseconds
283    pattern = >1
284    comment = Just for testing
285
286    Notice the pattern in this alert. It means that an alert will be triggered
287    as soon as a sample measurement has "ANY" delay, that is, more than one
288    millisecond. This is just for testing. In reality, you will want to create
289    an alert based on your observed baseline. For example, if your DNS servers'
290    delay suddendly goes from under 10 ms to over 100ms.
291
292    Next, be sure you have this test alert defined for some of your Targets.
293    You can either turn on alerts by defining alerts for a probe in
294    the /etc/smokeping/config.d/Probes file, or by individual Targets
295    entries.
296
297    In our case let's edit the Targets file and turn on alerts for our
298    DNS Latency checks.
299
300    $ sudo vi /etc/smokeping/config.d/Targets
301
302    Find the following section in the file:
303
304        +DNS
305        probe = DNS
306        menu = External DNS Check
307        title = DNS Latency
308
309        ++nsrc
310        host = nsrc.org
311
312    And add the following alerts line after "+++ nsrc"
313
314        +++nsrc
315        host = nsrc
316        alerts = anydelay
317
318    Save and exit from the file, then restart smokeping:
319
320    $ sudo service smokeping restart
321
322    Check your e-mail with mutt
323
324    $ mutt
325
326    (or check your RT queues)
327
328    And see if you have received alerts after 5 minutes.
329
3306. MultiHost Graphs
331
332    Once you have defined a group of hosts under a single probe type in your
333    /etc/smokeping/config.d/Targets file, then you can create a single graph
334    that will show you the results of all smokeping tests for all hosts that
335    you define. This has the advantage of letting you quickly compare, for
336    example, a group of hosts that you are monitoring with the FPing probe.
337
338    The MultiHost graph function in Smokeping is extremely picky - pay close
339        attention.
340
341    To create a MultiHost graph first edit the file Targets:
342
343    $ sudo vi Targets
344
345    If you had a section for the FPing probe defined that looked like this
346    (this is an example only - your Targets file may look different):
347
348        +Local
349        menu = Local
350        title = Local Network
351
352        ++LocalMachine
353        menu = Local Machine
354        title = This host
355        host = localhost
356
357        ++pc1
358        menu = pc1
359        title = pc1
360        host = pc1
361
362        ++pc2
363        menu = pc2
364        title = pc2
365        host = pc2
366
367        ++pc3
368        menu = pc3
369        title = pc3
370        host = pc3
371
372    Right now smokeping displays the results of the FPing probe for each
373    host defined in separate graphs. If you wish to see the results in a
374    single graph with multiple lines, then you would do this after the last
375    FPing probe host definition:
376
377        +MultiHostPCs
378        menu = MultiHost Ping
379        title = Consolidated Ping Response Time
380        host = /Local/LocalMachine /Local/pc1 /Local/pc2 /Local/pc3
381
382    (Note: if the lines get too long, you can have multiple lines for the
383    "host" entry by using the "\" character to indicate another line - ask about
384    this if you are unsure!)
385
386    Now save and exit the file Targets and restart smokeping:
387
388    $ sudo service smokeping restart
389
390    You should see a new graph under the "MultiHost Ping" menu in your
391    smokeping web interface. This graph will have different color lines
392    for each host you have defined.
393
394
3957. Slave instances - only done if we have the time.
396
397    This is a description only for informational purposes in case you wish
398    to attempt this type of configuration once the workshop is over.
399
400    The idea behind this is that you can run multiple smokeping instances
401    at multiple locations that are monitoring the same hosts and/or services
402    as your master instance. The slaves will send their results to the
403    master server and you will see these results side-by-side with your
404    local results. This allows you to view how users outside your network
405    see your services and hosts.
406
407    This can be a powerful tool for resolving service and host issues that
408    may be difficult to troubleshoot if you only have local data.
409
410    Graphically this looks this:
411
412          [slave 1]     [slave 2]      [slave 3]
413                |             |              |
414                +-------+     |     +--------+
415                        |     |     |
416                        v     v     v
417                        +---------------+
418                        |    master     |
419                        +---------------+
420
421    You can see example of this data here:
422
423    http://oss.oetiker.ch/smokeping-demo/
424
425    Look at the various graph groups and notice that many of the graphs
426    have multiple lines with the color code chart listing items such as
427    "median RTT from mipsrv01" - These are not MultiHost graphs, but rather
428    graphs with data from external smokeping servers.
429
430    To configure a smokeping master/slave server you can see the documentation
431    here:
432
433    http://oss.oetiker.ch/smokeping/doc/smokeping_master_slave.en.html
434
435    In addition, a sample set of steps for configuring this is available in
436    the file sample-smokeping-master-slave.txt which is available under the
437    attachments listing on the agenda page of the class wiki.
438</pre>
439</body>
440</html>