Bug0101

Description: can't get scanners working with darcs pyflag.

Using pyflag from the darcs repository, on Debian Lenny, can't get HTTP, webmail (maybe because of HTTP scanner) and MSN scanners working. The packet capture file from wireshark seems to load well into the VFS, then the filesystem scan shows the following errors in the pyflag console.

No results show up in the "Browse MSN Data" and the "Browse HTTP requests" won't show more than one HTTP request. These features work as expected on the pyflag Windows port, I get all the HTTP requests, hotmail messages and MSN chat using these scanners

Here are the errors:

31586(Debug): Opening Combined Stream S1458/1459 for MSN
31585(Debug): Openning Ichat|S1348/1349 for HTTP
31586(Critical Error): Scanner (<NetworkScanner.Scan instance at 0x9bda86c>) on Inode Ichat|S1458 Error: 'NoneType' object is not iterable
31586(Critical Error): Scanner (<NetworkScanner.Scan instance at 0x9bda86c>) on Inode Ichat|S1458 Error: 'NoneType' object is not iterable
  File "/tmp/pyflag/lib/python2.5/site-packages/pyflag/Scanner.py", line 512, in scanfile
    o.finish()
  File "/tmp/pyflag/lib/python2.5/site-packages/pyflag/plugins/NetworkForensics/NetworkScanner.py", line 113, in finish
    self.outer.process_stream(self.fd, self.factories)
  File "/tmp/pyflag/lib/python2.5/site-packages/pyflag/plugins/NetworkForensics/ProtocolHandlers/MSN.py", line 1940, in process_stream
    reverse_fd = self.fsfd.open(inode = reverse_inode)
  File "/tmp/pyflag/lib/python2.5/site-packages/pyflag/FileSystem.py", line 159, in open
    retfd = Registry.VFS_FILES.vfslist[part[0]](self.case, retfd, '|'.join(sofar))
  File "/tmp/pyflag/lib/python2.5/site-packages/pyflag/plugins/NetworkForensics/Reassembler.py", line 100, in __init__
    self.__dict__.update(row)
None


32313(Debug): Scanning file /bla/streams/2009-02-17/192.168.1.80-67.15.212.197/1200:80/combined (inode Ibla|S303/304)
32313(Critical Error): Scanner (<HTTP.Scan instance at 0xa601bec>) on Inode Ibla|S1057/1058 Error: local variable 'combined_inode' referenced before assignment
32313(Critical Error): Scanner (<HTTP.Scan instance at 0xa601bec>) on Inode Ibla|S1057/1058 Error: local variable 'combined_inode' referenced before assignment
  File "/tmp/pyflag/lib/python2.5/site-packages/pyflag/Scanner.py", line 512, in scanfile
    o.finish()
  File "/tmp/pyflag/lib/python2.5/site-packages/pyflag/plugins/NetworkForensics/NetworkScanner.py", line 113, in finish
    self.outer.process_stream(self.fd, self.factories)
  File "/tmp/pyflag/lib/python2.5/site-packages/pyflag/plugins/NetworkForensics/ProtocolHandlers/HTTP.py", line 376, in process_stream
    pyflaglog.log(pyflaglog.DEBUG,"Openning %s for HTTP" % combined_inode)
None

Logged: 2009-04-06
Logged By: flop
Assigned to: Mic
Status: Closed

Get back to the BugIndex

Hmm this seems to be because its unable to find the reverse stream - is it possible your data is only one way (i.e. only contains one side of the TCP conversation?)

The packet-captures tested are two-ways, some with wireshark protocol filters (eg. msnms - no TCP handshakes included then), and some without any filter. But you seem to be right about the reverse stream. While browsing the virtual filesystem, I only see "Forward" inodes, and no reverse ones. So this may be a problem with stream reassembling ?

I checked with my pyflag windows version, and I got forward as well as reverse streams inodes, with the same capture files !

Ok I think I found the bug - it was caused by use of aliased pointers which is apparently a no no. It tended to interact badly with gcc's optimizations. Please test the latest patch from darcs.

Thank you, your last commit fixed the bug, now it works ! I close this report (guess changing status to Closed is enough)

Bug0101

Search:

Toolbox