Tideway Community Forum

TPL: How to trigger each time a Host is scanned

2008-10-01T16:40:39+00:00

We’ve added another article to the Power Tips series to start a few simple tips if you are just starting out in pattern writing.

How to trigger each time a Host is scanned?

Short answer:

// The correct way to trigger each time a Host is scanned
  triggers
    on host := Host created, confirmed
  end triggers; 

TPL: model.Host gotchas

2008-09-24T09:51:27+00:00

A frequent gotcha I’ve seen in TPL is accidentally using model.Host() when you meant model.host() or host.attribute

I’ve written this up with examples in this article.

If you want the executive summary:

Any use of model.Host() in TPL should be avoided.

From the next release onwards the TPL compiler will warn you of this situation.

Playback Windows record data using Foundation

2008-06-05T15:28:13+00:00

The advice in this post is relevant for the 6.x range of Foundation. Additional techniques are available on the 7.x range.

You may decide to rescan a host with the information available in record data. This can be useful when you are consolidating multiple tideway appliances discovered (record) data into one appliance for reporting purpose.

The first thing you will do is change the discovery mode to playback in the UI

Method #1 - Using the Appliance Pool Data

1. Scan the Windows machines (eg. 172.17.4.46) with a Slave connected to an Appliance.
2. If you haven’t just scanned the host(s) and are loading data from another source you will only need the pool data sourced from the Appliance.
3. Create a .no-expiry file in that host’s pool data on the Appliance. To do this for all hosts do: find /usr/tideway/var/pool/ -type d -links 2 -exec touch {}/.no-expiry ;
4. At this time you should also confirm that the full set of pool data files are there (getInfo getInterfaceList getNames getNetworkConnectionList getPackageList getProcessList getProcessToConnectionMapping nmap-small), if not delete the pool data and rescan the host again. Make sure the pool-expiry isn’t set to -1 or something ridiculously low in ~/bin/tw_svc_discovery.
5. Go into the Tideway UI and delete the host nodes for the Windows machines.
6. Stop the Slave service on the Windows machine and stop the tideway service on the Appliance.
7. For post-6.2 Appliances change the Discovery Mode in Setup->Appliance->System Settings.
8. Scan the Windows IPs (eg. 172.17.4.46) in playback mode, there is no need to setup credentials.
9. Once finished playing back data run find /usr/tideway/var/pool/ -type f -name .no-expiry -exec rm -f {} ; to remove the .no-expiry files and restore normal pool data functionality.
Note: When you use the PDA (Playback Data Archive) way of gathering the pool data, it creates the .no-expiry files for you as well. There’s also a script called tw_pool2playback which is a command line tool that creates a PDA called PFI_Data (Primary Foundation Instance, it came out of the BAM Lite work) from all the current pool data.

Method #2 - Using Record Data on the Appliance and Slave

1. Scan the Windows machines (eg. 172.17.4.46) with a Slave connected to an Appliance.
2. Delete the host nodes for the Windows machines.
3. If you haven’t just scanned the host(s) and are loading data from another source you will need the record data sourced from the Appliance as well as the record data sourced from the Slave.
4. Stop the the Slave service on the Windows slave host and set it to playback mode by altering the CommandLine value in the HKEY_LOCAL_MACHINESOFTWARETidewayDISCOVERY<Slave Type> Registry key.
5. Appliance. For post-6.2 Appliances change the Discovery Mode in Setup->Appliance->System Settings.
6. Create a file called os_type with the contents Windows in each record data directory for a Windows host. This script automates this process.
7. Restart both services, make sure that the Appliance is connected to the Slave.
8. If you are using a Credential Slave make sure there are some credentials for the IP ranges you want to playback. These credentials can be anything - and username and password as long as the password isn’t blank. This is not required for AD/Workgroup Slaves.
9. Start Discovery and scan the host(s).

Method #3- Using existing appliance record data only

The below instructions are for Windows hosts. For Unix hosts you just need the record data on the Appliance and then you can set the Appliance to playback mode and do a scan.
1. Create record directories on the slave and appliance for the host IP you want to scan. (eg. var/record/172/17/4/46/)
2. Copy the record data to the slave.
3. Check if your record data has a file called ‘nmap’ if not copy from another set of record data and edit the ip address in the file. Or create a file called os_type as above.
4. Copy the nmap and nmap-small files to the record directory on your appliance.
5. With your Slave and Appliance set to playback mode scan the ip address of your record data. (eg. 172.17.4.46)

Sharing table lookups between TPL modules

2008-06-05T12:37:16+00:00

You may not know that you can share table definitions between modules. You can see examples of this in TKU (see the SupportingFiles.Tables module).

For example, define a module with a table definition:

tpl 1.0 module TEST1;

// Define a table to be shared in other modules
table LookupPath 1.0
  'sudo' -> ['/sbin/sudo', '/usr/local/bin/suexec'];
end table; 

This allows a value to be returned by the table based on a key (in this case a list of strings). In one or more other modules, “import” the table in module scope with the command:


from TEST1 import LookupPath 1.0;

and then the table can be used in a pattern this second module as normal:

tpl 1.0 module TEST2;

from TEST1 import LookupPath 1.0;

pattern TEST2 1.0
  '''
  Test pattern to use imported table
  '''
  
  overview
    tags TEST;
  end overview;

  triggers
    on device := DeviceInfo;
  end triggers;
  
  body
    host := model.host(device);
    if not host then
      log.error('No host node for: % device.name%');
      stop;
    end if;
    
    for val in LookupPath['sudo'] do
      log.info('>>> Possible path: % val%');
    end for;
  end body;
end pattern; 

Don’t forget regex.extractAll()

2008-06-06T10:26:27+00:00

When writing TPL you will probably end up using regex.extract() most frequently, however don’t forget to check the TPL guide for its friend, regex.extractAll() which is useful when you need to extract several data items matching a single pattern.

This example shows its use, for extracting nameserver out of a Unix hosts’s DNS config file:

tpl 1.0 module TEST;

pattern TEST 1.0
  '''
  Demo to show how to use extractAll
  '''
  
  overview
    tags TEST;
  end overview;

  triggers
    on device := DeviceInfo where os_class = 'UNIX';
  end triggers;
  
  body
    host := model.host(device);
    if not host then
      log.error('No host node for: % device.name%');
      stop;
    end if;
    
    file := discovery.fileGet(host, '/etc/resolv.conf');
    if file then
      nameservers := regex.extractAll(file.content, regex 'nnameservers+(S+)');
    
      for nameserver in nameservers do
        log.info('>>>> Detected nameserver % nameserver%');
      end for;
    end if;
  end body;
end pattern; 

Difference between fqdn and dns_domain in Foundation

2008-06-05T14:43:38+00:00

The fqdn is the host’s Fully Qualified Domain Name. Ie. The hostname with the domain that the host is in appended, this produces a resolvable DNS name. Eg. from inside the Tideway network you can see the mail server as just ‘mail’ (ie. ping mail) or as mail.tideway.com, but from outside it must be stated ‘fully-qualified’ as mail.tideway.com to be resolvable.

The dns_domain is from /etc/resolv.conf on the host and is the DNS domain to search in if none is specified. Ie. If I’m sitting on the Tideway network how does the DNS client on my machine know that when it gets a non-fully qualified name like ‘mail’ to turn it into ‘mail.tideway.com’. See man resolv.conf