Linux, Programming

If you want to convert a program to hexdecimal or binary out, here is two programs to do so.

First get your pre-reqs:

apt-get update
apt-get install python
apt-get install pip
pip install bitstring

Note: these programs would work outside of linux, just need to make sure you have the bitstring python module installed.

EXAMPLE OUTPUT

Imagine a source file, that will be our input file for the tohex/tobin programs, that looks like this:

text1
text2

binary output:

Converting to BINARY
Reading in test.txt
*** START ***
011101000110010101111000011101000011000100001010011101000110010101111000011101000011001000001010
*** THE END ***<span style="font-family: Lato, sans-serif; font-size: 16px; line-height: 1.5; background-color: #ffffff;"> </span>

hexadecimal output:

Converting to HEXADECIMAL / HEX
Reading in test.txt
*** START ***
74657874310a74657874320a
*** THE END ***

SIMPLE PROGRAMS

Now here is a simple version of converting to hex:

#!/bin/bash
# usage: ./tohex-simple.py <filename>
# converts filename to hex/hexidecimal output
import bitstring
import os.path
import sys
INPUTFILE = sys.argv[1]
BSTREAM = bitstring.ConstBitStream(filename=INPUTFILE)
BS = 8 # blocksize in bits, reading the file in packets of <BS> bit chunks
    try:
        while True:
            BPACKET = BSTREAM.read(BS)  ## READ IN BLOCKSIZE (in bits)
            sys.stderr.write(BPACKET.hex) ### prints BS_NUM bits in hex at a time
    except bitstring.ReadError:
       pass

And here is the simple version of converting to binary:

#!/bin/bash
# usage: ./tobin-simple.py <filename>
# converts filename to binary output
import bitstring
import os.path
import sys
INPUTFILE = sys.argv[1]
BSTREAM = bitstring.ConstBitStream(filename=INPUTFILE)
BS = 8 # blocksize in bits, reading the file in packets of <BS> bit chunks
    try:
        while True:
            BPACKET = BSTREAM.read(BS)  ## READ IN BLOCKSIZE (in bits)
            sys.stderr.write(BPACKET.bin) ### prints BS_NUM bits in hex at a time
    except bitstring.ReadError:
       pass

BLOCKSIZE INFO

Note about blocksize: the blocksize, BS, is set to 4096, I find thats a good amount to read in while keeping the program fast. I converted a 1MB file in a tenth of a second with blocksize of 4096 bits, but it took 4 seconds with a block size of 128 bits, and 17 seconds with a 32 bit block size. Reading in a byte at a time (blocksize is 8 bits) took 44 seconds to process the 1MB file.

Caveat: Had to drop the block size to 8, which is the size of a byte, or else this wouldn’t work on inputfiles that are less than the blocksize in size (as a ReadError would incur). Likewise for files that are not even blocksize integer size will get truncated at blocksize integers. With setting block size to 8 we fix that problem at the cost of performance

THE REAL PROGRAMS

Now if you want a fancy program, that can have helpful info, and print on the desktop pretty and can handle errors, then use the programs below.

Hexadecimal / hex version:

#!/usr/bin/python

### EXTRACTS HEX DATA AND PRINTS TO FILE ###
### stdout prints output can be saved to file###
### stderr prints info text, and newlines to pretty output ###

import bitstring
import os.path
import sys

__author__ = 'infotinks' ## optional line

def werr(txt): # write error - print to stderr
    sys.stderr.write(txt)

def wout(txt): # print without newline - print to stdout
    sys.stdout.write(txt)

def main(filename): # parse the file
    f = filename
    needfinalnewline=False # for printing newline on a special case (when stderr is redirect but stdout is not)
    bs = 8 # blocksize in bits, reading the file in packets of 128 BIT chunks
    werr("Converting to HEXADECIMAL / HEX\n")
    werr("Reading in %s\n" % f)
    if os.path.isfile(f):
        pass
    else:
        werr("ERROR: %s is not a file (or doesnt exist)\n" % f)
        return -1
    werr("*** START ***\n")
    bstream = bitstring.ConstBitStream(filename=f)
    try:
        while True:
            bpacket = bstream.read(bs)  ## READ IN BLOCKSIZE (in bits)
            if sys.stdout.isatty(): # stdout not redir but going to terminal so write to error so that order on screen looks correct (or else all stderr output is at the top)
                if sys.stderr.isatty():
                    werr(bpacket.hex) ### *** MAIN OUTPUT ***
                else:
                    wout(bpacket.hex) ### *** MAIN OUTPUT ***
                    needfinalnewline = True
            else: # stdout is being redirected, so write to it
                wout(bpacket.hex)
    except bitstring.ReadError:
       pass
    if needfinalnewline: # prints a newline at end, if all error output is redirected to a different file
        wout("\n")
    else:
        if sys.stdout.isatty(): # not redirected print like this so pretty
            werr("\n*** THE END ***\n")
        else: # stdout is being redirected so remove first newline to make pretty
            werr("*** THE END ***\n")

### THE START OF THE PROGRAM ###

try:
    file = sys.argv[1]
except:
    werr("*** Converts file to HEX ***\n")
    werr("ERROR: Need to give inputfile filename\n")
    werr("usage: %s inputfile\n" % sys.argv[0])
    werr("usage: %s inputfile > outputfile\n" % sys.argv[0])
    werr("usage: %s inputfile 2> /dev/null\n" % sys.argv[0])
else:
    try:
        main(file)
    except KeyboardInterrupt:
        werr("\n*** CANCELLED or ERROR ***\n")

Binary version:

#!/usr/bin/python

### EXTRACTS BINARY DATA AND PRINTS TO FILE ###
### stdout prints output can be saved to file###
### stderr prints info text, and newlines to pretty output ###

import bitstring
import os.path
import sys

__author__ = 'infotinks' ## optional line

def werr(txt): # write error - print to stderr
    sys.stderr.write(txt)

def wout(txt): # print without newline - print to stdout
    sys.stdout.write(txt)

def main(filename): # parse the file
    f = filename
    needfinalnewline=False # for printing newline on a special case (when stderr is redirect but stdout is not)
    bs = 8 # blocksize in bits, reading the file in packets of 128 BIT chunks
    werr("Converting to BINARY\n")
    werr("Reading in %s\n" % f)
    if os.path.isfile(f):
        pass
    else:
        werr("ERROR: %s is not a file (or doesnt exist)\n" % f)
        return -1
    werr("*** START ***\n")
    bstream = bitstring.ConstBitStream(filename=f)
    try:
        while True:
            bpacket = bstream.read(bs) ## READ IN BLOCKSIZE (in bits)
            if sys.stdout.isatty(): # stdout not redir but going to terminal so write to error so that order on screen looks correct (or else all stderr output is at the top)
                if sys.stderr.isatty():
                    werr(bpacket.bin) ### *** MAIN OUTPUT ***
                else:
                    wout(bpacket.bin) ### *** MAIN OUTPUT ***
                    needfinalnewline = True
            else: # stdout is being redirected, so write to it
                wout(bpacket.bin)
    except bitstring.ReadError:
       pass
    if needfinalnewline: # prints a newline at end, if all error output is redirected to a different file
        wout("\n")
    else:
        if sys.stdout.isatty(): # not redirected print like this so pretty
            werr("\n*** THE END ***\n")
        else: # stdout is being redirected so remove first newline to make pretty
            werr("*** THE END ***\n")

### THE START OF THE PROGRAM ###

try:
    file = sys.argv[1]
except:
    werr("*** Converts file to BINARY ***\n")
    werr("ERROR: Need to give inputfile filename\n")
    werr("usage: %s inputfile\n" % sys.argv[0])
    werr("usage: %s inputfile > outputfile\n" % sys.argv[0])
    werr("usage: %s inputfile 2> /dev/null\n" % sys.argv[0])
else:
    try:
        main(file)
    except KeyboardInterrupt:
        werr("\n*** CANCELLED or ERROR ***\n")

The end.

Leave a Reply

Your email address will not be published. Required fields are marked *