Files: exercises-smokeping.txt

File exercises-smokeping.txt, 10.7 KB (added by hervey, 9 years ago)

Smokeping Exercises in TEXT

Line 
1Registry Operations Curriculum
2Smokeping
3
4Notes:
5------
6* Commands preceded with "$" imply that you should execute the command as
7  a general user - not as root.
8* Commands preceded with "#" imply that you should be working as root.
9* Commands with more specific command lines (e.g. "RTR-GW>" or "mysql>")
10  imply that you are executing commands on remote equipment, or within
11  another program.
12
13Exercises
14----------
15
160. Log in to your PC or open a terminal window as the tladmain user.
17
18Once you are logged in you can continue with these exercises.
19
201. Install Smokeping
21
22        $ sudo apt-get install smokeping
23       
24
252. Initial Configuration
26
27        $ cd /etc/smokeping
28
29        No changes are necessary in this page. You can update the Smokeping
30        look and feel by editing /etc/smokeping/basepage.html
31
32        $ cd /etc/smokeping/config.d
33        $ ls -l
34
35        -rwxr-xr-x 1 root root  578 2010-02-26 01:55 Alerts
36        -rwxr-xr-x 1 root root  237 2010-02-26 01:55 Database
37        -rwxr-xr-x 1 root root  413 2010-02-26 05:40 General
38        -rwxr-xr-x 1 root root  271 2010-02-26 01:55 pathnames
39        -rwxr-xr-x 1 root root  859 2010-02-26 01:55 Presentation
40        -rwxr-xr-x 1 root root  116 2010-02-26 01:55 Probes
41        -rwxr-xr-x 1 root root  155 2010-02-26 01:55 Slaves
42        -rwxr-xr-x 1 root root 8990 2010-02-26 06:30 Targets
43 
44        The files you need to touch (at a minimum) are:
45
46        * Alerts
47        * General
48        * Probes
49        * Targets
50
51        Edit Alerts
52
53        $ sudo vi Alerts
54
55        Update the top of the file where it says:
56
57                *** Alerts ***
58                to = alertee@address.somewhere
59                from = smokealert@company.xy
60
61        to include a proper "to" and "from" field for your server.
62        Something like:
63
64                *** Alerts ***
65                to = tldadmin@localhost
66                from = smokeping-alert@localhost
67
68        If you were going to create tickets from Smokeping alerts
69        the "to" address would be an alias for the ticketing system.
70        We will do this a bit later.
71
72        Add a new alert for later use:
73
74                +rttbadstart
75                type = rtt
76                # in milliseconds
77                pattern = ==S,==U
78                priority = 1
79                comment = offline at startup
80
81        * "==S, ==U" means "at Startup" and "not Up"
82        * "priority = 1" means if multiple alerts are defined for a host
83          and multiple alerts match one the one with the highest priority
84          is executed.
85
86        Now save the file and exit, then edit the file General:
87
88        $ sudo vi General
89
90         Change the following lines:
91
92                owner
93                contact
94                cgiurl
95                        mailhost
96
97        Something like this should work:
98
99                owner    = AROC Class User
100                contact  = tldadmin@localhost
101                cgiurl   = http://localhost/cgi-bin/smokeping.cgi
102                mailhost = localhost
103
104        Now save the file and exit, then edit the file Probes:
105
106        $ sudo vi Probes
107
108        The current entry in Probes is fine, but if you wish to
109        use additional Smokeping checks you can add them in here
110        and you can specify their default behavior. You can do
111        this, as well, in the Targets file if you wish.
112
113        Here is an example of a Probes file that would specify
114        what to use to check for HTTP and DNS latency as well as
115        the FPing probe that is used for ping latency:
116
117                *** Probes ***
118
119                + FPing
120
121                binary = /usr/bin/fping
122
123                + EchoPingHttp
124
125                + DNS
126                binary = /usr/bin/dig
127                pings = 5
128                step = 180
129
130        Go ahead and update your Probes file with this information.
131        Then save the file and exit. And, now let's restart the
132        Smokeping service to verify that no mistakes have been made
133        before going any further:
134
135        $ sudo /etc/init.d/smokeping stop
136        $ sudo /etc/init.d/smokeping start
137
138        You could, also do:
139
140        $ sudo /etc/init.d/smokeping restart
141
142        or
143
144        $ sudo /etc/init.d/smokeping reload
145
146        to reload configuration changes. This should work in most
147        cases.
148
149
1503. Configure monitoring of devices
151
152        The majority of your time and work configuring Smokeping
153        will be done in the file /etc/smokeping/config.d/Targets.
154       
155        For this class please do the following:
156
157        Use the FPing probe to check:
158          - all the student NOC PCs
159          - classroom NOC
160          - switches
161          - routers:
162         
163          You can use the classroom Network Diagram on the classroom wiki (http://noc/) to
164          figure out addresses for each item, etc.
165
166        Create some hierarchy to the Smokeping menu for your
167        checks. Such as:
168
169                        PCs
170                Routers
171                Switches
172
173        Add a check for HTTP latency for all the classroom NOC PCs.
174        This will mean adding another category, such as:
175
176                NOCs HTTP
177
178        If you have time consider checking some machines that are
179        external to our classroom and the conference.
180
181        Look at additional Smokeping probes and consider implementing
182        some of them:
183
184        http://oss.oetiker.ch/smokeping/probe/index.en.html
185
186        As trying to explain all syntactical details of how the file
187        /etc/smokeping/config.d/Targets is used would require several
188        pages we will go through some examples in class, and you can
189        refer to the Smokeping configuration files that are in use on
190        the classroom NOC box by going to:
191
192        http://noc/configs/etc/smokeping
193        http://noc/configs/etc/smokeping/config.d
194
1954. Add DNS Latency Checks
196
197        You can check either or both internal or external names using
198        the DNS latency probe.
199
200        Add a menu hierarchy for DNS Latency. Check an external address
201        (nsrc.org) and an internal address (noc). This will look something
202        like this (in Targets):
203
204        $ sudo vi /etc/smokeping/config.d/Targets
205
206                ++ DNS
207                probe = DNS
208                menu = External DNS Check
209                title = DNS Latency
210
211                +++ nsrc
212                host = nsrc.org
213
214
215                +++ noc
216                host = noc.aroc
217
218        Exit and save your changes to the file Targets.
219
220        Restart Smokeping to see the changes:
221
222        $ sudo /etc/init.d/smokeping restart
223
224       
2255. Send Smokeping alerts to our Request Tracker Net queue
226
227        We've already set this up in RT and in /etc/aliases. You just
228        need to point Smokeping alerts to our RT instance. Edit the file
229        Alerts:
230
231        $ sudo vi /etc/smokeping/config.d/Alerts
232
233        And change:
234
235                to = tldadmin@localhost
236
237        to
238
239                to = net@localhost
240
241        Now whenever Smokeping sends an alert email with that alert text
242        will arrive to the Net queue in Request Tracker.
243
244        Next, be sure you have alerts defined for some of your Targets.
245        You can either turn on alerts by defining alerts for a probe in
246        the /etc/smokeping/config.d/Probes file, or by individual Targets
247        entries.
248
249        In our case let's edit the Targets file and turn on alerts for our
250        DNS Latency checks. In addition, if you add a DNS latency check for
251        a host that does not exist, then we can see a ticket being created
252        in RT.
253
254        $ sudo vi /etc/smokeping/config.d/Targets
255
256        Find the following section in the file:
257
258                ++ DNS
259                probe = DNS
260                menu = External DNS Check
261                title = DNS Latency
262
263                +++ nsrc
264                host = nsrc.org
265
266
267                +++ noc
268                host = noc.aroc
269
270        And, add the following host after "+++ noc"
271
272                +++ noexist
273                host = does.not.exist
274                alerts = rttbadstart
275
276        Save and exit from the file, then restart smokeping:
277
278        $ sudo /etc/init.d/smokeping restart
279
280        You will see an error message on the screen:
281
282                WARNING: Hostname 'does.not.exist' does currently not resolve to
283                an IPv6 or IPv4 address
284
285        This is to be expected as the host "does.not.exist" is not a valid
286        host. But, Smokeping still starts, and the rttbadstart Alert will
287        now send email to the Net queue for Request Tracker. If you open a
288        web browser to your RT instance:
289
290        http://localhost/rt/
291
292        and log in as "tldadmin" you will see a new ticket in the home screen
293        that has a subject of:
294
295        "[SmokeAlert] rttbadstart is active on
296        AROC.DNSProbe.RT-test"
297       
298
2996. MultiHost Graphs
300
301        Once you have defined a group of hosts under a single probe type in your
302        /etc/smokeping/config.d/Targets file, then you can create a single graph
303        that will show you the results of all smokeping tests for all hosts that
304        you define. This has the advantage of letting you quickly compare, for
305        example, a group of hosts that you are monitoring with the FPing probe.
306
307        To create a MultiHost graph first edit the file Targets:
308
309        $ sudo vi Targets
310
311        If you had a section for the FPing probe defined that looked like this
312        (this is an example only - your Targets file may look different):
313
314                + Local
315                menu = Local
316                title = Local Network
317
318                ++ LocalMachine
319                menu = Local Machine
320                title = This host
321                host = localhost
322
323                ++ pc1
324                menu = pc1
325                title = pc1
326                host = pc1
327
328                ++ pc2
329                menu = pc2
330                title = pc2
331                host = pc2
332
333                ++ pc3
334                menu = pc3
335                title = pc3
336                host = pc3
337
338        Right now smokeping displays the results of the FPing probe for each
339        host defined in separate graphs. If you wish to see the results in a
340        single graph with multiple lines, then you would do this after the last
341        FPing probe host definition:
342
343                + MultiHostPCs
344                menu = MultiHost Ping
345                title = Consolidated Ping Response Time
346                host = /Local/LocalMachine /Local/pc1 /Local/pc2 \
347                       /Local/pc3
348
349        Note how  you can have multiple lines for the "host" entry by using the
350        "\" character to indicate another line.
351
352        Now save and exit the file Targets and restart smokeping:
353
354        $ sudo /etc/init.d/smokeping restart
355
356        You should see a new graph under the "MultiHost Ping" menu in your
357        smokeping web interface. This graph will have different color lines
358        for each host you have defined.
359
360
3617. Slave instances
362
363        This is a description only for informational purposes in case you wish
364        to attempt this type of configuration once the workshop is over.
365
366        The idea behind this is that you can run multiple smokeping instances
367        at multiple locations that are monitoring the same hosts and/or services
368        as your master instance. The slaves will send their results to the
369        master server and you will see these results side-by-side with your
370        local results. This allows you to view how users outside your network
371        see your services and hosts.
372
373        This can be a powerful tool for resolving service and host issues that
374        may be difficult to troubleshoot if you only have local data.
375
376        Graphically this looks this:
377
378                  [slave 1]     [slave 2]      [slave 3]
379                        |             |              |
380                        +-------+     |     +--------+
381                                |     |     |
382                                v     v     v
383                                +---------------+
384                                |    master     |
385                                +---------------+
386
387        You can see example of this data here:
388
389        http://oss.oetiker.ch/smokeping-demo/
390
391        Look at the various graph groups and notice that many of the graphs
392        have multiple lines with the color code chart listing items such as
393        "median RTT from mipsrv01" - These are not MultiHost graphs, but rather
394        graphs with data from external smokeping servers.
395
396        To configure a smokeping master/slave server you can see the documentation
397        here:
398
399        http://oss.oetiker.ch/smokeping/doc/smokeping_master_slave.en.html
400
401        In addition, a sample set of steps for configuring this is available in
402        the file sample-smokeping-master-slave.txt.
403