Refine Forwarder Request Rates

The Stairwell forwarder is configured out of the box to be as lightweight as possible while still accomplishing the goals of providing unparalleled visibility into your environment. In some rare cases, it is desirable to change request rates to better match your environment. This feature is available in the following forwarder versions.

There are two major configurations you can change, which we've outlined below.

Repeat Sightings

The repeat sightings flag sets the time during which multiple sightings of the same file hash at the same file path on the same asset will be ignored. For example, if file with hash abc123 is sighted at C:/myDir/file.exe on asset ABC at 1 pm and this time is set to 6 hours, then additional sightings of the hash abc123 at the path C:/myDir/file.exe on asset ABC will be ignore until 7 pm. If the file changes contents or location at all, or is seen on a different asset, a new sighting will always be recorded. Note that this sighting data is only used to update the last seen time in the stairwell application and will not prevent the location or bytes of the file from being recorded.

This flag may be set between 1 hour and 24 hours, with a default setting of 6 hours. In general, setting the flag to a lower value will increase the number of requests the forwarder makes to stairwell, as well mildly increase the amount of CPU used by the forwarder, while making the last seen time more accurate. Setting the flag to a larger value will do the opposite. The main use of the last seen time in the stairwell application is for doing contemporaneous analysis, which looks for activity around the time that known malicious files were active, and having this value too high can make that more difficult. In practice, a value of 6 hours limits resource usage without hampering contemporaneous analysis.

Batch Upload Rate Limit

The batch upload rate limit sets the amount of time that sightings are batched before being uploaded. For example, if the flag is set to 20 seconds and a first sighting is made at 1 pm, all sightings between 1 pm and 20 seconds after 1 pm will be sent as a single request to stairwell at 20 seconds after 1 pm. Note that there are very rare cases where sightings requests can be sent more frequently than this if the sightings would otherwise be lost.

This flag may be set between 1 second and 1 minute, with a default setting of 20 seconds. In general, setting the flag to a lower rate will increase the number of requests the forwarder makes to stairwell, as well as mildly increasing the amount of CPU used by the forwarder. Setting the flag to a larger value will do the opposite. The main risk of setting the flag to a higher value is that events may possibly be lost if the forwarder loses connectivity or crashes during batching time or if the files sighted change very rapidly during that time. In practice, a value of 20 seconds or less should be sufficient to guard against sighting loss while maintaining reasonable request rates.