Nessus with Nikto – Running out of memory

Kind of an annoying problem, but sometimes nikto runs out of control. This is made worse by nessus, which can have a lot of nikto instances running at once.

Dec 29 13:03:10 mopey-macky kernel: [72355.838027] Free swap = 0kB
Dec 29 13:03:10 mopey-macky kernel: [72355.838031] Total swap = 5855684kB
Dec 29 13:03:10 mopey-macky kernel: [72355.866431] 1048576 pages RAM
Dec 29 13:03:10 mopey-macky kernel: [72355.866436] 38328 pages reserved
Dec 29 13:03:10 mopey-macky kernel: [72355.866440] 9361 pages shared
Dec 29 13:03:10 mopey-macky kernel: [72355.866444] 1000493 pages non-shared
Dec 29 13:03:10 mopey-macky kernel: [72355.866451] Out of memory: kill process 6730 (run-mozilla.sh) score 665297 or a child
Dec 29 13:03:10 mopey-macky kernel: [72355.866556] Killed process 6734 (thunderbird-bin)

Yes, that was fun, randomly killed processes because I’m out of memory. some instances of nikto were taking 2gb of memoy and spidering infinitely over these dynamic pages.

To fix, I added a stupid watchdog script.

#!/usr/bin/python

import subprocess
import time

#percent of memory the nikto is taking
MAXMEMPERCENT = 13
#time is in hours
MAXTIME = 1
#time in seconds to check
SLEEPYTIME = 60
lfile=open("./nikto_wd.log", "a")

while 1:
  p1 = subprocess.Popen(["ps", "aux"], stdout=subprocess.PIPE)
  p2 = subprocess.Popen(["grep", "nikto"], stdin=p1.stdout, stdout=subprocess.PIPE)
  output = p2.communicate()[0].split("n")

  for line in output:
    #print line
    thisline = line.split()
    try:
      if ("/usr/bin/perl" in thisline[10] and thisline[3] != "" and thisline[9] != ""):
        memusage =  float(thisline[3])
        hours = int(thisline[9][0])
        #process needs to be killed 
        if int(hours) > MAXTIME or float(memusage) > MAXMEMPERCENT:
          print "die, zombie scum", thisline
          lfile.write("die, zombie scum " + str(thisline) + "n")
          subprocess.call(["kill", thisline[1]])
    except IndexError:
      pass
  lfile.flush()
  time.sleep(SLEEPYTIME)

proxychains – handy tool!

proxychains is a pretty amazing tool available at http://proxychains.sourceforge.net/. It is a versitile proxy tool. So folks like me, who would like the source IPs to be from a proxy, or multiple proxys. For me, the main uses are proxying gui port scan stuff like nessus and proxying tor.

Proxying port scans can be handy if you want the address to come from something else. For example, you might have an ssh server somewhere that you’d like to scan from. Or you might want to port scan through tor. To porscan through an ssh server

ssh -D 2323 mysshserver

#edit /etc/proxychains.conf  so socks4 is set to 2323

#socks4  127.0.0.1 2323

proxychains nmap -T4…

then all nmap traffic will appear to come from your ssh server. Very cool! In addition, you can set up a tor proxy, haver proxychains point to it from proxychains.conf, and launch your program similarly using proxychains. This has the advantage of having everything go through tor. So if you wanted you could port scan through tor.

A usually more legitimate use would be to launch firefox using proxychains through tor. This is superior to simply setting the proxy through ff itself because when ff sets a local proxy there is still dns leakage, potential flash leakage etc. If it is launched through the proxy, all children of the process go through tor.

mycontroller – done

This was my final architecture project from a couple years ago. I had several posts in here about it, so I figured I’d post the final as well.

The goal of this project was to integrate all parts covered throughout the lab. Similar to lab7, lab8 fetches microcode from a second memory device. Extending lab7, however, it also performs very basic operations, including add, eq, nop, ld, skipz, and halt. There are 4 physical registers, which will be referred to as 00, 01, 10, and 11.

lab8report.pdf <– Write up

Logicv12 <— Multimedia Logic Source

lab8.zip <– contains the default program loaded into ram, etc

The circuitry is pretty simple – most the hard work was in decompiling the microops. Here are some shots of the circuitry.

Screenshot-1

Auto Pw Change

I had to change this script a lot, so take with a grain of salt. That said, we changed about 1000 LOCAL passwords in a couple hours – which would have really taken all day and been more boring.

#!/usr/bin/python

import pexpect

#most likely should be first for speed
passlist = ["pass1", "pass2", "pass3"]
#most critical should be listed in file first for speed
user="root"
newpass="newpass"

#open hosts file
hostfile=open("hosts.txt", "r")

for host in hostfile:
  host = host.strip()
  changeSuccesful = False
  #need to find the currpass to change it
  #so auth by key may not be ideal in this case
  p = pexpect.spawn("ssh " + user + "@" + host + " passwd"
  
  #try block so it doesn't crash the program
  try:
    #different systmes vary with exact text
    conn_result = p.expect(["assword:", pexpect.EOF, "Are you sure you want to continue"])
    if conn_result == 2:
      print "accepting public key for ", host
      p.sendline("yes")
      conn_result = p.expect(["assword:", pexpect.EOF])
    if conn_result == 0:
      for password in passlist:
        print "tryin password for ", host
        p.sendline(password)
        pass_result = p.expect(["denied", "current.*assword:", "new.*assword", pexpect.EOF])
        if pass_result == 1:
          p.sendline(password)
          p.expect("new.*assword:")
        #this should execute if a key OR password was accepted
        if pass_result == 1 or pass_result == 2:
          p.sendline(newpass)
          p.expect("new.*assword:")
          p.sendline(newpass)
          changeSuccesful = True
          print "Succesful pwchange: host "+ host 
          break
    if not changeSuccesful:
      print "UnSuccesful pwchange: host "+ host 
  except:
    print "Uncaught exception: host "+ host 


Where was the Hacker in the Room for X-FRAME-OPTIONs?

Update: don’t worry, they took care of it – but as of Feb 2012 only IE:  http://blogs.msdn.com/b/ieinternals/archive/2010/03/30/combating-clickjacking-with-x-frame-options.aspx

Or maybe where was the Dev in the room?

Imagine you’re sitting at a computer logged into your favorite website, lolcats, and you click on a shady link while logged in. There are a lot of attack scenarios that start this way.

Browsers have a cross domain policy that prevent the shady site from accessing any of your lolcats cookies, which typically contain your authentication tokens. But BY DESIGN, the shady site is certainly allowed to do requests (eg POST or GET) across domains. So there’s nothing to prevent the shady site from framing lolcats in an invisible iframe and having you play an animated whack-a-mole game, clicking exactly where the invisible lolcats site is having you inadvertently  rank the attacker’s disgusting kitten as cute. This is performed with your very own account, because the cookies are all legitimate from being logged in. This makes all your lolcats buddies laugh at your bad taste.

This is contrived, sure. But tools are getting a lot better to automate this sort of thing. You don’t need a whack-a-mole game to do clickjacking anymore, there are whole Javascript frameworks to automate everything. In fact, the attack is as easy as just getting them to visit the shady site, no clicks required. One such tool is:

http://www.contextis.co.uk/resources/tools/clickjacking-tool/

Currently clickjacking defense is treated somewhat less importantly than XSRFf, but the fact is, the surface area for a clickjacking attack is basically identical to XSRF. Both are confused deputy problems.

One of the first ways to defend against this attack was frame busting scripts, which are snippets of Javascript and HTML to try to make it so websites can’t be invisibly framed. These are notoriously difficult, and can vary from application to application. To deal with this problem, Spencer Low went into a cave of solitude for some time and came up with a pretty good framebusting solution. Unfortunately, it turns out whatever frame busting solution you have is circumventable using IE’s XSS filter in IE8 or IE9. Details are

http://www.darkreading.com/security/vulnerabilities/showArticle.jhtml?articleID=225200337

So there is only one defense that really works to defend against clickjacking, and this is X-FRAME-OPTIONs.  X-FRAME-OPTIONs is a newish header designed by Microsoft that’s now included in all modern browsers (chrome, Firefox, Safari, in addition to IE8 and IE9). What X-FRAME-OPTIONs does is set a header that says this response cannot be framed except under certain circumstances. There does need to be work on server applications to add this header, but I believe this is fundamentally the right approach to stop clickjacking. You don’t want lolcats framed by the shady site? Just configure lolcats to put the X-FRAME-OPTIONs header in the response and it cannot be framed.

Here’s the problem, X-FRAME-OPTIONs has three options: ALLOW, SAMEORIGIN, and DENY. That’s it. And they do exactly what you’d think they do.

What if, by design, I want my application to be framed by something in a different domain? There are a lot of legitimate circumstances this happens, and they pop up all the time in Online Services.

As a security industry, what we do right now in these situations is say there’s no good and easy defense. Web applications are sometimes vulnerable to clickjacking because the fact is there’s nothing good we can really do on these edge case scenarios. We can (and do) develop frame busting scripts that takes a lot of work, probably have holes, and are certainly bypassible in IE8 and IE9 due to the XSS protection unless we just explicitly switch off the XSS protection. It’s an active problem to determine if switching off XSS protection in order to hopefully develop a script that might with enough effort prevent clickjacking is worth it.

I wonder why when designing X-FRAME-OPTIONs it wasn’t just designed as a whitelist solution. Instead of only having the ALLOW, SAMEORIGIN, and DENY options, it could have a list of domains that are allowed to frame the content. It would make so much more sense to explicitly allow domains that are allowed to frame our application instead of being arbitrarily restricted to sameorigin. It would give us flexibility while at the same time allowing us to be safe.

I applaud the IE team for coming up with X-FRAME-OPTIONs in the first place, and for other browsers to adopt it. It can protect 95% of sites from clickjacking. I just wish that we didn’t have to have vulnerable sections of applications just because of the limitation of options.

8-queens problem hill climbing python implementation

It was written in an AI book I’m reading that the hill-climbing algorithm finds about 14% of solutions. I implemented a version and got 18%, but this could easily be due to different implementations – like starting in random columns rather than random places on the board, and optimizing per column. Anyway, here is the program.

README

This program is a hillclimbing program solution to the 8 queens problem. The algorithm is silly in some places, but suits the purposes for what I was working on I think. It was tested with python 2.6.1 with psyco installed. If big runs are being tried, having psyco may be important to maintain sanity, since it will speed things up significanlty. Otherwise, you may want to stick to –numrun being less than around 50.

The board is simply defined as a two dimensional list, with the occupied elements stored as “Q” and empty emements as 0. The initial board is generated by picking a random row and column to place a queen, although the class structure allows for predefined boards to be manually passed in. If the spot on the board is occupied, then another spot is randomly chosen.

Violations are calculated by iterating through every queen and checking horizontally, vertically, and diagonally for other queens. Each violation is totalled up, and at the end they are divided by 2 since violations were overcounted. This could certainly be optimized further.

The hill solution works by checking every possible single move and returning the best of these. Obviously, this could also be improved upon. The book’s algorithm (which was not available while programming this) simply attempts to move every space within a column rather than every open spot on the board – which would speed up the process by an order of magnitude and also decrease the likelihood of finding a solution by a small percentage. Also, it appears that the random initial state only contains one queen per column, which is also different from this implementation. The assignment specification mentions a randomly generated board, which is what this implementation was based on. If an implementation closer to that of the book is desired, please let me know, as it would only be a minor adjustment.

With this algorithm, every queen on the board tries to move to every spot on the board, and violations are re-calculated. A move with the least violations is chosen and the process repeats until there is no improvement. It there is no improvement after every queen has had a go, there is no solution found and the algorithm returns. If there is an improvement, the algorithm continues for another go-around.

The biggest run so far is just 1000 nodes. It returned 175 successes, which is fairly close to the book’s given percentage or .14.

Here is sample usage:

mopey-mackey:hillclimb user$ python eight_queen.py –help
Usage: eight_queen.py [options]

Options:
-h, –help show this help message and exit
-q, –quiet Don’t print all the moves… wise option if using large
numbers
–numrun=NUMRUN Number of random Boards

mopey-mackey:hillclimb user$ python eight_queen.py –numrun=1000 –quiet
Total Runs: 1000
Total Success: 175
Success Percentage: 0.175
Average number of steps: 3.83

mopey-mackey:hillclimb user$ python eight_queen.py
====================
BOARD 0
====================
Board Violations 7
0 0 Q 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 Q 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 Q 0 0 0
Q Q 0 0 0 0 Q 0
0 0 0 0 0 0 0 0
0 0 0 0 Q 0 0 0

Board Violations 4
0 0 Q 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 Q 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 Q 0 0 0
Q Q 0 0 0 0 0 0
0 0 0 0 0 0 0 Q
0 0 0 0 Q 0 0 0

Board Violations 3
0 0 Q 0 0 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 0 Q 0 0
0 0 0 0 0 0 0 0
0 0 0 0 Q 0 0 0
Q Q 0 0 0 0 0 0
0 0 0 0 0 0 0 Q
0 0 0 0 Q 0 0 0

Board Violations 2
0 0 Q 0 0 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 0 Q 0 0
0 Q 0 0 0 0 0 0
0 0 0 0 Q 0 0 0
0 Q 0 0 0 0 0 0
0 0 0 0 0 0 0 Q
0 0 0 0 Q 0 0 0

NO SOLUTION FOUND
Total Runs: 1
Total Success: 0
Success Percentage: 0.0
Average number of steps: 4.0

mopey-mackey:hillclimb user$ python eight_queen.py –numrun=4
====================
BOARD 0
====================
Board Violations 3
0 0 Q 0 0 0 0 0
Q 0 0 0 Q 0 0 0
0 0 0 0 0 0 0 Q
0 0 0 Q 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 Q 0
0 Q 0 0 0 0 0 0
0 0 0 0 0 0 0 Q

Board Violations 1
0 0 Q 0 0 0 0 0
Q 0 0 0 Q 0 0 0
0 0 0 0 0 0 0 Q
0 0 0 Q 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 Q 0
0 Q 0 0 0 0 0 0
0 0 0 0 0 Q 0 0

Board Violations 0
0 0 Q 0 0 0 0 0
0 0 0 0 Q 0 0 0
0 0 0 0 0 0 0 Q
0 0 0 Q 0 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 0 0 Q 0
0 Q 0 0 0 0 0 0
0 0 0 0 0 Q 0 0

SOLUTION FOUND
====================
BOARD 1
====================
Board Violations 8
Q 0 0 0 0 0 0 Q
0 0 0 0 0 0 Q 0
0 0 Q 0 Q 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 Q 0 0
0 0 0 0 0 0 0 0
0 Q 0 0 Q 0 0 0
0 0 0 0 0 0 0 0

Board Violations 5
Q 0 0 0 0 0 0 Q
0 0 0 0 0 0 Q 0
0 0 Q 0 Q 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 Q 0 0
0 0 0 0 0 0 0 0
0 0 0 0 Q 0 0 0
0 Q 0 0 0 0 0 0

Board Violations 3
0 0 0 0 0 0 0 Q
0 0 0 0 0 0 Q 0
0 0 Q 0 Q 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 0 Q 0 0
0 0 0 0 0 0 0 0
0 0 0 0 Q 0 0 0
0 Q 0 0 0 0 0 0

Board Violations 2
0 0 0 0 0 0 0 Q
0 0 0 0 0 0 Q 0
0 0 Q 0 0 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 0 Q 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 Q 0 0 0
0 Q 0 0 0 0 0 0

NO SOLUTION FOUND
====================
BOARD 2
====================
Board Violations 5
0 Q 0 Q 0 0 0 0
0 0 0 0 0 Q 0 Q
Q 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
Q 0 0 0 0 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 0 0 Q 0
0 0 0 0 0 0 0 0

Board Violations 3
0 Q 0 Q 0 0 0 0
0 0 0 0 0 Q 0 Q
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
Q 0 0 0 0 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 0 0 Q 0
0 0 0 0 Q 0 0 0

Board Violations 2
0 Q 0 Q 0 0 0 0
0 0 0 0 0 Q 0 0
0 0 0 0 0 0 0 Q
0 0 0 0 0 0 0 0
Q 0 0 0 0 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 0 0 Q 0
0 0 0 0 Q 0 0 0

Board Violations 1
0 Q 0 Q 0 0 0 0
0 0 0 0 0 Q 0 0
0 0 0 0 0 0 0 Q
0 0 Q 0 0 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 Q 0
0 0 0 0 Q 0 0 0

Board Violations 0
0 Q 0 0 0 0 0 0
0 0 0 0 0 Q 0 0
0 0 0 0 0 0 0 Q
0 0 Q 0 0 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 Q 0 0 0 0
0 0 0 0 0 0 Q 0
0 0 0 0 Q 0 0 0

SOLUTION FOUND
====================
BOARD 3
====================
Board Violations 5
0 0 0 Q Q 0 0 0
0 0 0 0 0 0 0 Q
0 0 0 0 0 0 0 0
0 0 0 0 0 Q 0 0
0 0 0 0 0 0 0 Q
0 Q 0 0 0 0 0 0
0 0 0 0 Q 0 0 0
0 0 Q 0 0 0 0 0

Board Violations 3
0 0 0 0 Q 0 0 0
0 0 0 0 0 0 0 Q
0 0 0 Q 0 0 0 0
0 0 0 0 0 Q 0 0
0 0 0 0 0 0 0 Q
0 Q 0 0 0 0 0 0
0 0 0 0 Q 0 0 0
0 0 Q 0 0 0 0 0

Board Violations 1
0 0 0 0 Q 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 Q 0 0 0 0
0 0 0 0 0 Q 0 0
0 0 0 0 0 0 0 Q
0 Q 0 0 0 0 0 0
0 0 0 0 Q 0 0 0
0 0 Q 0 0 0 0 0

Board Violations 0
0 0 0 0 Q 0 0 0
Q 0 0 0 0 0 0 0
0 0 0 Q 0 0 0 0
0 0 0 0 0 Q 0 0
0 0 0 0 0 0 0 Q
0 Q 0 0 0 0 0 0
0 0 0 0 0 0 Q 0
0 0 Q 0 0 0 0 0

SOLUTION FOUND
Total Runs: 4
Total Success: 3
Success Percentage: 0.75
Average number of steps: 4.0

And here is the source

#!/usr/bin/python

import random,sys,copy
from optparse import OptionParser
try:
  import psyco
  psyco.full()
except ImportError:
  pass

"""
cowboy code, but seems to work
USAGE: python prog <numberruns=1> <verbocity=False>
"""

class board:
  def __init__(self, list=None):
    if list == None:
      self.board = [[0 for i in range(0,8)] for j in range(0,8)]
      #initialize queens at random places
      for i in range(0,8):
        while 1:
          rand_row = random.randint(0,7)
          rand_col = random.randint(0,7)
          if self.board[rand_row][rand_col] == 0:
            self.board[rand_row][rand_col] = "Q"
            break
    #TODO raise errors if board is not right format or dimension
  #define how to print the board
  def __repr__(self):
    mstr = ""
    for i in range(0,8):
      for j in range(0,8):
        mstr = mstr + str(self.board[i][j]) + " "
      mstr = mstr + "n"
    return (mstr)

class queens:
  def __init__(self, numruns, verbocity, passedboard=None):
    #TODO check options
    self.totalruns = numruns
    self.totalsucc = 0
    self.totalnumsteps = 0
    self.verbocity = verbocity
    for i in range(0,numruns):
      if self.verbocity == True:
        print "===================="
        print "BOARD",i
        print "===================="
      self.mboard = board(passedboard)
      self.cost = self.calc_cost(self.mboard)
      self.hill_solution()

  def hill_solution(self):
    while 1:
      currViolations = self.cost
      self.getlowercostboard()
      if currViolations == self.cost:
        break
      self.totalnumsteps += 1
      if self.verbocity == True:
        print "Board Violations", self.calc_cost(self.mboard)
        print self.mboard
    if self.cost != 0:
      if self.verbocity == True:
        print "NO SOLUTION FOUND"
    else:
      if self.verbocity == True:
        print "SOLUTION FOUND"
      self.totalsucc += 1
    return self.cost

  def printstats(self):
    print "Total Runs: ", self.totalruns
    print "Total Success: ", self.totalsucc
    print "Success Percentage: ", float(self.totalsucc)/float(self.totalruns)
    print "Average number of steps: ", float(self.totalnumsteps)/float(self.totalruns)

  def calc_cost(self, tboard):
    #these are separate for easier debugging
    totalhcost = 0
    totaldcost = 0
    for i in range(0,8):
      for j in range(0,8):
        #if this node is a queen, calculate all violations
        if tboard.board[i][j] == "Q":
          #subtract 2 so don't count self
          #sideways and vertical
          totalhcost -= 2
          for k in range(0,8):
            if tboard.board[i][k] == "Q":
              totalhcost += 1
            if tboard.board[k][j] == "Q":
              totalhcost += 1
          #calculate diagonal violations
          k, l = i+1, j+1
          while k < 8 and l < 8:
            if tboard.board[k][l] == "Q":
              totaldcost += 1
            k +=1
            l +=1
          k, l = i+1, j-1
          while k < 8 and l >= 0:
            if tboard.board[k][l] == "Q":
              totaldcost += 1
            k +=1
            l -=1
          k, l = i-1, j+1
          while k >= 0 and l < 8:
            if tboard.board[k][l] == "Q":
              totaldcost += 1
            k -=1
            l +=1
          k, l = i-1, j-1
          while k >= 0 and l >= 0:
            if tboard.board[k][l] == "Q":
              totaldcost += 1
            k -=1
            l -=1
    return ((totaldcost + totalhcost)/2)

  #this function tries moving every queen to every spot, with only one move
  #and returns the move that has the leas number of violations
  def getlowercostboard(self):
    lowcost = self.calc_cost(self.mboard)
    lowestavailable = self.mboard
    #move one queen at a time, the optimal single move by brute force
    for q_row in range(0,8):
      for q_col in range(0,8):
        if self.mboard.board[q_row][q_col] == "Q":
          #get the lowest cost by moving this queen
          for m_row in range(0,8):
            for m_col in range(0,8):
              if self.mboard.board[m_row][m_col] != "Q":
                #try placing the queen here and see if it's any better
                tryboard = copy.deepcopy(self.mboard)
                tryboard.board[q_row][q_col] = 0
                tryboard.board[m_row][m_col] = "Q"
                thiscost = self.calc_cost(tryboard)
                if thiscost < lowcost:
                  lowcost = thiscost
                  lowestavailable = tryboard
    self.mboard = lowestavailable
    self.cost = lowcost

if __name__ == "__main__":

  parser = OptionParser()
  parser.add_option("-q", "--quiet", dest="verbose",
                   action="store_false", default=True,
                   help="Don't print all the moves... wise option if using large numbers")

  parser.add_option("--numrun", dest="numrun", help="Number of random Boards", default=1,
                   type="int")

  (options, args) = parser.parse_args()

  mboard = queens(verbocity=options.verbose, numruns=options.numrun)
  mboard.printstats()

Follow

Get every new post delivered to your Inbox.