new CSV scripts added

Author: ehanson8

README.md

@@ -1,6 +1,9 @@
 # text-audio-image-processing
 Scripts for processing audio and images
 
+#### [combineCSVs.py](combineCSVs.py)
+Based on user input, combines several CSV file that have the same specified base file name followed by sequential numbers (e.g 'report1.csv', 'report2.csv', 'report3.csv'). A file name suffix may added if one exists after the file number (e.g. 'report1edited.csv', 'report2edited.csv', 'report3edited.csv'). Used to combine files split by [splitCSV.py](splitCSV.py).
+
 #### [dhashImageComparison.py](dhashImageComparison.py)
 Based on user input, creates dhashes for all of the image files in the specified directory and then compares them using a BK-tree and creates a CSV file of all dhashes matches that are below the specified threshold (e.g. '40' means the dhashes are 40% different and 60% similar).
 
@@ -13,6 +16,9 @@ Based on a specified file and a specified threshold (e.g. '90' means the strings
 #### [stringComparisonFromCSVOldAndNew.py](stringComparisonFromCSVOldAndNew.py)
 Based on a specified files of new and old strings and a specified threshold (e.g. '90' means the strings are 90% similar and 10% different), compares each string against every other string in the new strings file, identifies all strings with a similarity above the specified threshold, and prints it to a new CSV file. It also compares each string to a CSV file of old strings that have previously received an authorized form (e.g. adding new name headings to an existing authority file)
 
+#### [splitCSV.py](splitCSV.py)
+Based on user input, splits the specified CSV file into separate CSV files with specified number of rows. The header row is repeated in each new file. The files can be combined later with [combineCSVs.py](combineCSVs.py).
+
 #### [transcribeAudioFile.py](transcribeAudioFile.py)
 Generates a rough, unformatted transcript of a specified MP3 using the free Wit ([https://wit.ai/](https://wit.ai/)) speech-to-text API. The script requires a secrets.py file in the same directory that must contain the following text:
 ```

combineCSVs.py

@@ -0,0 +1,33 @@
+import csv
+import argparse
+from pathlib import Path
+
+parser = argparse.ArgumentParser()
+parser.add_argument('-f', '--baseFileName', help='the base file name for the files to be combined. optional - if not provided, the script will ask for input')
+parser.add_argument('-s', '--suffix', help='the suffix that exists after the file number in the file name. optional - if not provided, the script will ask for input')
+args = parser.parse_args()
+
+if args.baseFileName:
+    baseFileName = args.baseFileName
+else:
+    baseFileName = raw_input('Enter the base file name for the files to be combined: ')
+if args.suffix:
+    suffix = args.suffix
+else:
+    suffix = raw_input('Enter the suffix that exists after the file number in the file name: ')
+
+f = csv.writer(open(baseFileName + 'Combined.csv', 'wb'))
+
+fileNum = 1
+file = baseFileName +str(fileNum) + suffix + '.csv'
+
+while Path(file).is_file() == True:
+    print file
+    with open(file) as csvfile:
+        reader = csv.reader(csvfile)
+        if fileNum > 1:
+            reader.next()
+        for row in reader:
+            f.writerow(row)
+    fileNum += 1
+    file = baseFileName +str(fileNum) + suffix + '.csv'

splitCSV.py

@@ -0,0 +1,34 @@
+import csv
+import argparse
+
+parser = argparse.ArgumentParser()
+parser.add_argument('-f', '--file', help='the CSV file to split. optional - if not provided, the script will ask for input')
+parser.add_argument('-n', '--num', help='the number of rows to include in each file. optional - if not provided, the script will ask for input')
+args = parser.parse_args()
+
+if args.file:
+    file = args.file
+else:
+    file = raw_input('Enter the CSV file to split: ')
+if args.num:
+    num = args.num
+else:
+    num = raw_input('Enter the number of rows to include in each file: ')
+
+with open(file) as csvfile:
+    reader = csv.DictReader(csvfile)
+    header = reader.fieldnames
+
+baseFileName = file.replace('.csv','')
+num = int(num)
+
+csvfile = open(file).readlines()
+filenum = 1
+for i in range(len(csvfile)):
+    if i % num == 0:
+        f = open(baseFileName + str(filenum) + '.csv', 'wb')
+        if filenum != 1:
+            f.write(str(header).replace('[','').replace(']','').replace('\'',''))
+            f.write('\n')
+        f.writelines(csvfile[i:i+num])
+        filenum += 1