Asked  7 Months ago    Answers:  5   Viewed   43 times

I have a Java program that reads some text from a file, line by line, and writes new text to an output file. But not all the text I write to my BufferedWriter appears in the output file after the program has finished. Why is that?

The details: the program takes a CSV text document and converts it into SQL commands to insert the data into a table. The text file has more than 10000 lines which look similar to following:

2007,10,9,1,1,1006134,19423882

The program seems to work fine except it just stops in the file randomly half way through creating a new SQL statement having printed it into the SQL file. It looks something like:

insert into nyccrash values (2007, 1, 2, 1, 4, 1033092, 259916);
insert into nyccrash values (2007, 1, 1, 1, 1, 1020246, 197687);
insert into nyccrash values (2007, 10, 9, 1

This happens after about 10000 lines but several hundred lines before the end of the file. Where the break happens is between a 1 and a ,. However, the characters doesn't seem important because if I change the 1 to a 42 the last thing written to the new file is 4, which is cutting off the 2 from that integer. So it seems like the reader or writer must just be dying after writing/reading a certain amount.

My Java code is as follows:

import java.io.*;

public class InsertCrashData
{
    public static void main (String args[])
    {
        try
        {   
            //Open the input file.
            FileReader istream = new FileReader("nyccrash.txt");
            BufferedReader in = new BufferedReader(istream);
            //Open the output file.
            FileWriter ostream = new FileWriter("nyccrash.sql");
            BufferedWriter out = new BufferedWriter(ostream);
            String line, sqlstr;

            sqlstr = "CREATE TABLE nyccrash (crash_year integer, accident_type integer, collision_type integer, weather_condition integer, light_condition integer, x_coordinate integer, y_coordinate integer);nn"; 
            out.write(sqlstr);

            while((line = in.readLine())!= null)
            {
                String[] esa = line.split(",");
                sqlstr = "insert into nyccrash values ("+esa[0]+", "+esa[1]+", "+esa[2]+", "+esa[3]+", "+esa[4]+", "+esa[5]+", "+esa[6]+");n";
                out.write(sqlstr);
            }
        }
        catch(Exception e)
        {
            System.out.println(e);
        }
    }
}

 Answers

37

You need to close your OutputStream which will flush the remainder of your data:

out.close();

The default buffer size for BufferedWriter is 8192 characters, large enough to easily hold hundreds of lines of unwritten data.

Tuesday, June 1, 2021
 
TheCarver
answered 7 Months ago
74

Put the word.txt directly as a child of the project root folder and a peer of src

Project_Root
    src
    word.txt

Disclaimer: I'd like to explain why this works for this particular case and why it may not work for others.

Why it works:

When you use File or any of the other FileXxx variants, you are looking for a file on the file system relative to the "working directory". The working directory, can be described as this:

When you run from the command line

C:EclipseWorkspaceProjectRootbin > java com.mypackage.Hangman1

the working directory is C:EclipseWorkspaceProjectRootbin. With your IDE (at least all the ones I've worked with), the working directory is the ProjectRoot. So when the file is in the ProjectRoot, then using just the file name as the relative path is valid, because it is at the root of the working directory.

Similarly, if this was your project structure ProjectRootsrcword.txt, then the path "src/word.txt" would be valid.

Why it May not Work

For one, the working directory could always change. For instance, running the code from the command line like in the example above, the working directory is the bin. So in this case it will fail, as there is not binword.txt

Secondly, if you were to export this project into a jar, and the file was configured to be included in the jar, it would also fail, as the path will no longer be valid either.

That being said, you need to determine if the file is to be an embedded-resource (or just "resource" - terms which sometimes I'll use interchangeably). If so, then you will want to build the file into the classpath, and access it via an URL. First thing you would need to do (in this particular) case is make sure that the file get built into the classpath. With the file in the project root, you must configure the build to include the file. But if you put the file in the src or in some directory below, then the default build should put it into the class path.

You can access classpath resource in a number of ways. You can make use of the Class class, which has getResourceXxx method, from which you use to obtain classpath resources.

For example, if you changed your project structure to ProjectRootsrcresourcesword.txt, you could use this:

InputStream is = Hangman1.class.getResourceAsStream("/resources/word.txt");
BufferedReader reader = new BufferedReader(new InputStreamReader(is));

getResourceAsStream returns an InputStream, but obtains an URL under the hood. Alternatively, you could get an URL if that's what you need. getResource() will return an URL

For Maven users, where the directory structure is like src/main/resources, the contents of the resources folder is put at the root of the classpath. So if you have a file in there, then you would only use getResourceAsStream("/thefile.txt")

Tuesday, June 1, 2021
 
Saxophlutist
answered 7 Months ago
47

This has to do with character encoding. Computers only deal with numbers. To store text, the characters in the text have to be converted to and from numbers, using some scheme. That scheme is called the character encoding. There are many different character encodings; some of the well-known standard character encodings are ASCII, ISO-8859-1 and UTF-8.

In the first example, you read all the bytes (numbers) in the file and then convert them to characters by passing them to the constructor of class String. This will use the default character encoding of your system (whatever it is on your operating system) to convert the bytes to characters.

In the second example, where you use Files.lines(...), the UTF-8 character encoding will be used, according to the documentation. When a sequence of bytes is found in the file that is not a valid UTF-8 sequence, you'll get a MalformedInputException.

The default character encoding of your system may or may not be UTF-8, so that can explain a difference in behaviour.

You'll have to find out what character encoding is used for the file, and then explicitly use that. For example:

String content = new String(Files.readAllBytes(Paths.get("_template.txt")),
        StandardCharsets.ISO_8859_1);

Second example:

Stream<String> lines = Files.lines(Paths.get("_template.txt"),
        StandardCharsets.ISO_8859_1);
Wednesday, July 21, 2021
 
Anele
answered 5 Months ago
82

You need to combine the LS and MS values bitwise, not arithmetically.

Try:

        ulong high = 30136437;
        unchecked
        {
            int low = -2138979250;
            uint uLow = (uint)low;
            high = high << 32;
            Date dt = DateTime.FromFileTime((long) (high | (ulong)uLow));
        }

Or any of the following should work too:

long highBits = time.dwHighDateTime;     
highBits = highBits << 32;     

return DateTime.FromFileTimeUtc(highBits + (long) (uint) time.dwLowDateTime); 

return DateTime.FromFileTimeUtc(highBits | (long) (uint) time.dwLowDateTime); 

return DateTime.FromFileTimeUtc(highBits + ((long)low & 0xFFFFFFFF))

return DateTime.FromFileTimeUtc(highBits | ((long)low & 0xFFFFFFFF))

You can get away with adding rather than a bitwise-or if you are sure the values are positive (and have no bits in common). But bitwise-or expresses the intent better.

Tuesday, August 10, 2021
 
Andres
answered 4 Months ago
53

The WriteModule function is static within lib/Bitcode/Writer/BitcodeWriter.cpp, which means it's not there for outside consumption (you can't even access it).

The same file has another function, however, called WriteBitcodeToFile, with this interface:

/// WriteBitcodeToFile - Write the specified module to the specified output
/// stream.
void llvm::WriteBitcodeToFile(const Module *M, raw_ostream &Out);

I can't imagine a more convenient interface. The header file declaring it is ./include/llvm/Bitcode/ReaderWriter.h, by the way.

Thursday, August 12, 2021
 
christina
answered 4 Months ago
Only authorized users can answer the question. Please sign in first, or register a free account.
Not the answer you're looking for? Browse other questions tagged :  
Share