A recently disclosed Black Hat hacker technique for reverse engineering intrusion prevention system (IPS) data poses a “serious risk” for thousands of enterprises, Gartner has warned.

The analyst firm’s warning comes after a speaker at the recent Black Hat Briefings conference in Las Vegas demonstrated a method of reverse-engineering IPS signatures for zero-day vulnerabilities. The demonstration used signatures from 3Com's TippingPoint IPS, but Gartner notes that there is “an implication” that all IPS vendor's signatures are at risk.

Paul E. Proctor, research vice president at Gartner, explained that enterprises use IPS technologies, which interpret external files containing signature definitions, to protect against the exploitation of vulnerabilities. However, when these patterns contain signatures for zero-day vulnerabilities, hackers can use this data to create exploit code based on vulnerabilities for which no protection exists. They can also use the signature file to write an exploit that bypasses the zero-day signature undetected, Proctor warned.

“Reverse-engineering code using interactive disassemblers such as DataRescue's IDA Pro Disassembler & Debugger is now a mainstream hacker activity, and a market has formed for the purchase of zero-day vulnerabilities. The rising value of these vulnerabilities represents a growing threat — both to the integrity of their own IPS protections and in the possibility of their becoming a source of compromised zero-day vulnerabilities — that enterprises should consider in their risk assessments,” Proctor said.

He added that the problem is not unique to TippingPoint IPS. On 22 May 2007, 3Com discontinued shipping Zero Day filters (signatures) in its regular updates. These filters are now available only after the request and the requesting party are verified and a strict nondisclosure agreement is signed.

Enterprises using IPSs from any vendor should, according to Proctor, check with their vendors to determine whether they provide zero-day signatures and, if so, hold them accountable for providing and disclosing protections that are in place for these signatures.

To help protect against this threat, Gartner urges IPS vendors to increase the complexity of reverse-engineering signature files that contain zero-day-vulnerability signatures.