Submitted by admin on Sat, 03/06/2010 - 00:57
Posted in
File Splitter Utility in C#
I had to come across a situation where I had to cut a large single .csv file containing 2.2 Million records into smaller files. Initially I thought of using already available split tool to do it, but then decided to write my own File Splitter tool in C#. Thats how I came up with this tool. The File Splitter Tool takes an input file and splits it into multiple files each containing the number of records you specify. Here is the source code for the tool.
Note:
You can download the Tool and the complete source code at the end of this post. To use the tool run it from the command prompt and provide the first parameter i.e the name of the input file you want to split. Other three parameters are optional.
Description:
The File Splitter tool accepts four parameters
- Input FileName (required)
- Output Path (optional)
- Number of records/lines in each file (optional)
- Prefix for the split files (optional)
The code uses StreamReader and StreamWriter classes to be memory efficient. It reads in the source file and splits the content of the file line by line and creates multiple files. Its very memory efficient.
In case of an Exception, the tool logs the error details in a log file which gets created if it does not already exist. If the log file already exists, the error details are appended to the existing log file
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Diagnostics;
namespace FileSplitter
{
class Program
{
static void Main(string[] args)
{
/*arguments
1. inputFileName
2. outputPath
3. Number of Lines in each file
4. File Prefix for Split Files
*/
StreamWriter f = null;
try
{
//***************************************************************************************
//Command line Arguments START
//***************************************************************************************
string inputFileName;
string outputPath = @""; //Default output folder
string filePrefixName = "new-"; //default Prefix
int fileSize = 100; //default = 100
if (args.Length == 0) //If no Argumments provided
{
Console.WriteLine("Please Enter the Name of the InputFile to Split");
return;
}
else if (args.Length == 1) //If Just one argument provided
{
inputFileName = args[0].ToString();
}
else if (args.Length == 2) //First two arguments provided
{
inputFileName = args[0].ToString();
outputPath = args[1].ToString();
}
else if (args.Length == 3) //First three arguments provided
{
inputFileName = args[0].ToString();
outputPath = args[1].ToString();
fileSize = Convert.ToInt32(args[2].ToString());
}
else
{
inputFileName = args[0].ToString();
outputPath = args[1].ToString();
fileSize = Convert.ToInt32(args[2].ToString());
filePrefixName = args[3].ToString();
}
//Command line Arguments END
//***************************************************************************************
//***************************************************************************************
//Read the File into Stream START
//***************************************************************************************
//As I am using the StreamReader within a using statement I dont have to worry about closing it manually in the Finally block.
using (StreamReader r = new StreamReader(inputFileName))
{
string record;
//***************************************************************************************
//Split the File START
//***************************************************************************************
int counter = 0;
int FNum = 1;
string FName = FNum.ToString() + ".csv";
FName = outputPath + filePrefixName + FName;
f = new StreamWriter(FName);
while ((record = r.ReadLine()) != null)
{
f.WriteLine(record);
counter++;
if (counter == fileSize)
{
FNum++;
FName = FNum.ToString() + ".csv";
FName = outputPath + filePrefixName + FName;
f = new StreamWriter(FName);
counter = 0;
}
}
f.Close();
//Split the File END
//***************************************************************************************
}
//***************************************************************************************
//Read the File into Stream END
//***************************************************************************************
}
catch (Exception ex)
{
string errorMsg;
errorMsg = "ERROR: " + ex.Message.ToString() + DateTime.Now;
using (FileStream file = new FileStream("FileSplitterLog.txt", FileMode.Append, FileAccess.Write))
{
StreamWriter streamWriter = new StreamWriter(file);
streamWriter.WriteLine(errorMsg);
streamWriter.Close();
}
}
finally
{
if (f != null)
{
f.Close();
}
}
}
}
}
| Attachment | Size |
|---|---|
| FileSplitter.zip | 2.51 KB |
| FileSplitter Source Code.zip | 20.63 KB |