Thursday, 12 September 2013

Break a text file into smaller chunks in python

Break a text file into smaller chunks in python

I have a text file (in MBs) and I want to break it into chunks(in KBs). I
am simulating the file transfer behaviour over a network. So far I was
able to make chunks according to the number of lines(seprated by '\n')
inputed by user like this
def make_chunks(fname):
ifile = file(fname,'rb')
file_iter = iter(ifile)
args = [file_iter] * 10 # No of lines you want to have in one chunk
chunks = list(izip_longest(fillvalue = None, *args))
But the chunks are now of different sizes.How would I make chunks of equal
size(say 4KB)

No comments:

Post a Comment