SmartCSV.fx version 0.4

Posted on July 30, 2016

New features for the application.

Improved performance and some nice additional features to help the user finding the errors.

Background validation with sidebar

To support the user in finding the validation errors, I added a sidebar with error marker and mouse over error explanations. This is something IntelliJ IDEA uses heavily in their IDE. When something is wrong, a red marker is shown on the side and a click on the marker scrolls the table directly to the error line.

In the first versions, I realized that the validation stopped the application from working, when there are some thousand lines to check.

Some little changes let the application perform well and be responsive all the time.

Avoid firing too many change actions

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

for (int lineNumber = 0; lineNumber < rows.size(); lineNumber++) {
    CSVRow row = rows.get(lineNumber);

    for (String column: row.getColumns().keySet()) {
        CSVValue value = row.getColumns().get(column).getValue();

        ValidationError validationError = validator.isValid(column, value.getValue(), lineNumber);
        if (validationError != null) {
            addValidationError(validationError);
            value.setValidationError(validationError);
        } else {
            value.setValidationError(null);
        }
    }
}

I use an ObservableList to store the error information for each line. And the sidebar has a listener on this list to know, when to redraw. The validation loops over all lines of the csv file and adds immediately the error information to the ObservableList. As you can imagine, this fires a lot of listeners, when there are a lot of errors. Changing the code to collect all the errors in a separate list and set the whole list at once sped up the validation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

List<ValidationError> errors = new ArrayList<>();

for (int lineNumber = 0; lineNumber < rows.size(); lineNumber++) {
    for (String column: row.getColumns().keySet()) {
        CSVValue value = row.getColumns().get(column).getValue();
        ValidationError validationError = validator.isValid(column, value.getValue(), lineNumber);
        if (validationError != null) {
            errors.add(validationError);
            value.setValidationError(validationError);
        } else {
            value.setValidationError(null);
        }
    }
}

validationError.setAll(errors);

With that optimization, the application worked again but wasn’t responsive during the validation circle.

Avoid doing time consuming stuff in FX-Thread

This should be a no brainer. To let the application become responsive again, I had to put the validation code into a separate thread. This is realized by using javafx.concurrent.Service.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

private static class RevalidationService extends Service<List<ValidationError>> {

    private Validator validator;
    private List<CSVRow> rows;
    private String[] header;

    public void setValidator(Validator validator) {
        this.validator = validator;
    }

    public void setRows(List<CSVRow> rows) {
        this.rows = rows;
    }

    public void setHeader(String[] header) {
        this.header = header;
    }

    @Override
    protected Task<List<ValidationError>> createTask() {
        return new Task<List<ValidationError>>() {
            @Override
            protected List<ValidationError> call() throws Exception {
                List<ValidationError> errors = new ArrayList<>();

                // do the validation

                return errors;
            }
        };
    }
}

Dialog to change the preferences

To support the different preferences a csv file can have, there is a dialog to change them. The preferences were persisted in the users home directory. Next time opening the application will restore the preferences.

Edit column validations

As not every user is familiar with json configurations, I thought supporting a dialog to configure the column validations would be a good thing.

A right click on the column header opens a context menu, with currently just one action. Opening the validation editor for the underlying column.

With the dialog the validations can be activated and deactivated and also the settings of the different checks can be entered directly. For the groovy check SmartCSV.fx also supports syntax highlighting.

With that feature, it should be easy for non developers to generate their own validation file for a csv file.

Generate an empty validation configuration

A normal csv workflow starts with an existing csv file. To support the validation, there must be an existing validation configuration file. As written in the validation configuration editor section, not all users are familiar with that technical stuff. To make the creation of the validation configuration easier, there is a new toolbar button. When clicking this button, an empty validation configuration is generated with a header validation based on the current csv file.

After generating the empty configuration, the user can edit the column validations and save the new validation configuration into a file. As long as the csv file format does not change, the configuration is ok. When changing the csv file format (more columns, changed header names or changed column order) the user has to edit the configuration file manually or start with an empty again.

Links

• GitHub - Page
• GitHub - Sources
• Wiki & Documentation
• JavaFX.ninja