Make Your Own Cluster Computer (Part 2)

davisde
[vc_row][vc_column width=”2/3″][vc_separator][venera_framed_image content_type=”video” css_animation=”appear” frame_type=”browser” slider_engine=”flexslider” video_link=”https://www.youtube.com/watch?v=lHmFRlETTcQ” browser_url=”https://www.youtube.com/watch?v=lHmFRlETTcQ”][vc_separator][/vc_column][vc_column width=”1/3″][/vc_column][/vc_row][vc_row][vc_column width=”2/3″][vc_tabs][vc_tab title=”About This Project” tab_id=”1402713028-1-39e9a4-2f88″][vc_column_text]

65_cluster_comp2_thThis is part two of the series that shows you how to make a cluster computer using Raspberry Pi’s!

You can watch Part 1 here.

[/vc_column_text][/vc_tab][vc_tab title=”Parts and Downloads” tab_id=”1402713028-2-39e9a4-2f88″][vc_column_text]

Acquiring the Parts
Here’s what you will need:

  1. 2 or more Raspberry Pi’s
  2. SD cards for each Pi
  3. Power Cables for each Pi
  4. Powered USB Hub (optional)
  5. Networking Cables
  6. A Hub or a Router

TOTAL COST: ~$100.00

[/vc_column_text][/vc_tab][vc_tab title=”Code” tab_id=”1402753910272-3-8″][vc_column_text]

Installing and Configuring Raspbian

  • Follow the software configuration steps shown in this video and burn the image file to an SD card for each computer in your cluster.
  • For each Raspberry Pi in your cluster, insert an imaged SD card, connect them all to the same router or switch, and then connect a power source to each one.
[tabby title=”Terminal Commands”]
    • Install nmap
      • Sudo apt-get update
      • Sudo apt-get install nmap
    • Get current IP
      • Ifconfig
    • Scan subnet for Pi’s
      • Sudo nmap -sn 192.168.1.*
    • Run test file
      • Mpiexec -n 1 hostname
    • Make test folder and file
      • Mkdir mpi_test
      • Cd mpi_test
      • Nano machinefile
        • 192.168.1.2
        • 192.168.1.3
        • 192.168.1.4
        • 192.168.1.5
      • Mpiexec -f machinefile -n 4 hostname
    • Add keys to all pi’s
      • PI01
        • Ssh-keygen
        • Cd ~
        • Cd .ssh
        • Cp id_rsa.pub pi01
        • Ssh pi@192.168.1.3
      • PI02
        • Ssh-keygen
        • Cd .ssh
        • Cp id_rsa.pub pi02
        • Scp 192.168.1.2:/home/pi/.ssh/pi01 .
        • Cat pi01 >> authorized_keys
        • Exit
      • PI03 (ssh pi@192.168.1.5)
        • Ssh-keygen
        • Cd .ssh
        • Cp id_rsa.pub pi03
        • Scp 192.168.1.2:/home/pi/.ssh/pi01 .
        • Cat pi01 >> authorized_keys
        • Exit
      • PI04
        • Ssh-keygen
        • Cd .ssh
        • Cp id_rsa.pub pi04
        • Scp 192.168.1.2:/home/pi/.ssh/pi01 .
        • Cat pi01 >> authorized_keys
        • Exit
      • PI01
        • Scp 192.168.1.3:/home/pi/.ssh/pi02
        • Cat pi02 >> authorized_keys
    • Run new machinefile
      • Cd ~
      • Nano machinefile
        • 192.168.1.2
      • Mpiexec -f machinefile -n 4 hostname
    • Run Python file
      • mpiexec -n 5 python demo/helloworld.py
      • Mpiexec -f machinefile -n 5 python /home/pi/build/mpi4py/demo/md5_dict_attack.py
    • Download and unzip file
    • Copy to all computers
      • Scp -r python_test 192.168.1.3:/home/pi
      • Mpiexec -f machinefile -n 5 python python_test/md5_attack.py
[/vc_column_text][/vc_tab][vc_tab title=”Important Links” tab_id=”1402753981900-3-10″][vc_column_text]

Help support my channel:

http://www.patreon.com/tinkernut

Follow Tinkernut!

Google +

Facebook

Twitter

[/vc_column_text][/vc_tab][/vc_tabs][/vc_column][vc_column width=”1/3″][/vc_column][/vc_row]

4 Comments

  1. pacmandu
    pacmandu
    September 9, 2015 at 4:20 am

    Your python script isn’t actually using MPI. You need to actually import MPI library and use their API’s for distributing tasks. All you are doing is running the same script on the Pi’s and not getting any more speed than just one Pi.

    • pacmandu
      pacmandu
      September 9, 2015 at 4:21 am

      from mpi4py import MPI

  2. pacmandu
    pacmandu
    September 9, 2015 at 4:27 am

    Here is the actual code for an MPI md5 python cracker. https://gist.githubusercontent.com/zooty/110cb97dcbeb17210765/raw/b83a30d382869e35b33f5f39be97d73ee1071e70/cracker.py

    • pacmandu
      pacmandu
      September 9, 2015 at 4:44 am

      Sorry but I still did not link to the correct MPI python code but you need to setup your code to know to send each word from the dictionary to one of the nodes in your cluster to hash and compare and then when one of the nodes reports back that it has a match you stop execution across all nodes. That unfortunately is not what your code is doing. When I build my cluster I will post a correct version that uses MPI to distribute the MD5 crack process.

Leave a Reply