A data matrix implementation 1

Though, as I wrote before, it's very hard to come up a universal dynamic data structure to fit all needs, sometimes we still need a somewhat general dynamic data structure due to requirements. For example, the spreadsheet could have as many columns during runtime, we can't determine the number of columns during compile time.

I have a very bad example of an implementation at work, it has some 3000 lines of code. I think it's for me to think about how it ends up like this and what are the better alternatives. Another motivation is still from the blog, I want to see how close I can get to ultimate - universal dynamic data structure, though I don't believe we can reach there. But the old saying says, you can't reach the North Star, but it always points to the right direction.

So here is my journey.

It's a data matrix, it has headers that we call fields, it has rows and columns. Each column has same data type.  This class is going across wires as well. And it has 3000 lines, for God's sake.

• stripped out the code for memory allocation, replace them with Collection classes
• stripped out the type meta data, removed all the type-safe getters/setters, they should be on the caller side, the meta data should be in the headers
• stripped out the methods related to cell/row/column computations. This should be a dedicated class later on since the same kind of logic spreads in a few places.
• stripped out the toXml() and other output methods.

After consolidate method signatures, The new class is like this

java 代码

import java.util.List;
import java.util.Map;
import java.util.ArrayList;
import java.util.HashMap;
/**
 * A Data matrix class, mainly for weak typed data, e.g., a table in swing.
 * Implementation reference, TableModel, DefaultTableModel.
 *
 * Note that the header type Field is really used as a flag, except the
 * hashcode used in the Map class.
 */
public class DataMatrix
{
    private List columns; // column values
    private List fields;  // field values, order is significant.

    private Map fieldToColumns; // what to do with duplicated fields.

    public DataMatrix()
    {
        fields = new ArrayList();
        columns = new ArrayList();
        fieldToColumns = new HashMap();
    }

    public DataMatrix(List fields)
    {
        this.fields = new ArrayList(fields);
        columns = new ArrayList(fields.size());
        fieldToColumns = new HashMap(fields.size());
    }

    public DataMatrix(DataMatrix dataMatrix)
    {
        this.columns = new ArrayList(dataMatrix.columns);
        this.fields = new ArrayList(dataMatrix.fields);
        this.fieldToColumns = new HashMap(dataMatrix.fieldToColumns);
    }

    //-----------------------------------------------------------------------
    // field operations
    //-----------------------------------------------------------------------
    public List getColumnFields() { return fields; }

    public void setColumnFields(List fields) { this.fields = new ArrayList(fields); }

    public Field getField(int col) { return (Field)fields.get(col); }

    public int getColumnNumber(Field field)
    {
        Integer index = (Integer)fieldToColumns.get(field);
        // should throw a cumstomized exception
        if (index == null) throw new IllegalArgumentException("field not found: " + field.toString());

        return index.intValue();
    }

    //-----------------------------------------------------------------------
    // column operations
    //-----------------------------------------------------------------------
    public int getNumberOfColumns() { return columns.size(); }

    public List getColumn(int col) { return (List) columns.get(col); }

    public List getColumn(Field key)
    {
        return (List) columns.get(getColumnNumber(key));
    }

    public void addColumn(Field field, List column)
    {
        checkNullAndEqualSize(field, column);

        fieldToColumns.put(field, new Integer(fields.size()));
        fields.add(field);
        columns.add(new ArrayList(column));
    }

    public void insertColumn(int position, Field field, List column)
    {
        checkNullAndEqualSize(field, column);

        fieldToColumns.put(field, new Integer(position));
        fields.add(position, field);
        columns.add(position, column);
    }

    private void checkNullAndEqualSize(Field field, List column)
    {
        if (field == null) throw new IllegalArgumentException("field is null");

        if (column == null) throw new IllegalArgumentException("column is null");

        if (columns.size() > 0)
        {
            List existingRow = (List)columns.get(0);
            if (existingRow.size() != column.size())
            {
                throw new IllegalArgumentException("column size doesn't match: "
                        + " existing column size=" + existingRow.size()
                        + " to be added column size=" + column.size());
            }
        }
    }

    public void deleteColumn(int position)
    {
        fieldToColumns.remove(fields.get(position));
        fields.remove(position);
        columns.remove(position);
    }

    public void deleteColumn(Field field)
    {
        Integer index = (Integer)fieldToColumns.get(field);
        if (index == null) return; // no such field here, do nothing

        fieldToColumns.remove(field);
        fields.remove(index.intValue());
        columns.remove(index.intValue());
    }

    // reorder the data matrix with the given fields.
    // if there is extra fields, throws exception
    public void reorder(Field[] fieldsInNewOrder)
    {
        // check fields to see whether they are the same.

        List newFields = new ArrayList();
        List newColumns = new ArrayList();
        Map newFieldToColumns = new HashMap();
        for (int i=0; i
        {
            newFields.add(fieldsInNewOrder[i]);
            int index = ((Integer)fieldToColumns.get(fieldsInNewOrder[i])).intValue();
            newColumns.add(columns.get(index));
            newFieldToColumns.put(fieldsInNewOrder, new Integer(i));
        }

        this.fields = newFields;
        this.columns = newColumns;
        this.fieldToColumns = newFieldToColumns;
    }

    //-----------------------------------------------------------------------
    // row operations
    //-----------------------------------------------------------------------
    public int getNumberOfRows()
    {
        if (columns.size() == 0) return 0;

        List firstColumn = (List)columns.get(0);
        return firstColumn.size();    // should optimize this - trade speed with space.
    }

    public List getRow(int row)
    {
        List ret = new ArrayList();
        for (int i=0; i
        {
            List columnList = (List) columns.get(i);
            ret.add(columnList.get(row));
        }

        return ret;
    }

    public void addRow(List row)
    {
        checkSizeMatch(row);

        for (int i=0; i
        {
            List columnList = getExistingOrNewColumn(i);
            columnList.add(row.get(i));
        }
    }

    public void insertRow(int position, List row)
    {
        checkSizeMatch(row);

        for (int i=0; i
        {
            List columnList = getExistingOrNewColumn(i);
            columnList.add(position, row.get(i));
        }
    }

    private void checkSizeMatch(List row)
    {
        if (row == null)
        {
            throw new IllegalArgumentException("row is null");
        }

        if (fields.size() == 0)
        {
            throw new IllegalStateException("set the fields before adding rows");
        }

        // not empty and not equals
        if (columns.size() != 0 && row.size() != columns.size())
        {
            throw new IllegalArgumentException("mismatched column size: " +
                    "datamatrix num of columns=" + columns.size() + " to be "
                    + "added row size=" + row.size());
        }
    }

    private List getExistingOrNewColumn(int i)
    {
        List columnList;
        if (i >= columns.size()) // empty columns
        {
            columnList = new ArrayList();
            columns.add(columnList);
        }
        else // existing
        {
            columnList = (List) columns.get(i);
        }
        return columnList;
    }

    // normal, in a data matrix, we need to loop from n-1 to 0
    // in order to use this in a loop.
    public void deleteRow(int rowNumber)
    {
        for (int i=0; i
        {
            List columnList = (List) columns.get(i);
            columnList.remove(rowNumber);
        }
    }

    //-----------------------------------------------------------------------
    // element operations
    //-----------------------------------------------------------------------
    // before you call these two methods, make sure you do populate the data.
    // Otherwise you will get an IndexOutOfBoundsException.
    public void setCellValueAt(int row, int col, Object value)
    {
        if (col >= columns.size())
        {
            throw new IllegalArgumentException("try to set value at an uninitialized cell");
        }

        ((List)columns.get(col)).set(row, value);
    }

    public Object getCellValue(int row, int col)
    {
        if (col >= columns.size())
        {
            throw new IllegalArgumentException("try to get value at an uninitialized cell");
        }

        List column = (List)columns.get(col);
        if (column == null || row >= column.size())
        {
            throw new IllegalArgumentException("try to get value at an uninitialized cell");
        }
        return ((List)columns.get(col)).get(row);
    }

    //-----------------------------------------------------------------------
    // matrix operations
    //-----------------------------------------------------------------------
    // append rows at the bottom
    public void concatenateRows(DataMatrix dataMatrix)
    {
        // check same number of columns first.
        if (columns.size() != dataMatrix.getNumberOfColumns())
        {
            throw new IllegalArgumentException("Two matrices have different "
               + "number of columns: " + columns.size() + " != " +
               dataMatrix.getNumberOfColumns());
        }

        for (int j=0; j
        {
            ((List)columns.get(j)).addAll(dataMatrix.getColumn(j));
        }
    }

    // append columns at the right end
    public void concatenateColumns(DataMatrix dataMatrix)
    {
        // check both matrices have the same number of rows.
        if (this.getNumberOfRows() != dataMatrix.getNumberOfRows())
        {
            throw new IllegalArgumentException("Two matrices have different "
               + "number of rows: " + getNumberOfRows() + " != " +
               dataMatrix.getNumberOfRows());
        }

        int startCol = columns.size();
        for (int j=0; j
        {
            fieldToColumns.put(dataMatrix.getField(j), new Integer(j + startCol));
            fields.add(dataMatrix.getField(j));
            columns.add(dataMatrix.getColumn(j));
        }
    }

    // split the datamatrix to two which are the two sides of this field.
    // The specified field could be on the right/left/not included.
    public DataMatrix[] splitFrom(Field field)
    {
        int colNum = getColumnNumber(field);

        DataMatrix leftMatrix = new DataMatrix();
        DataMatrix rightMatrix = new DataMatrix();

        for (int i=0; i
        {
            leftMatrix.addColumn((Field)fields.get(i), (List)columns.get(i));
        }

        for (int i=colNum; i
        {
            rightMatrix.addColumn((Field)fields.get(i), (List)columns.get(i));
        }
        return new DataMatrix[] {leftMatrix, rightMatrix};
    }

    public String toString()
    {
        String ret = "";

        if (fields == null) return ret;

        // output fields first
        for (int i=0; i
        {
            ret += fields.get(i).toString() + " | ";
        }
        ret += "\n";

        if (columns == null) return ret;

        // output rows
        for (int i=0; i
        {
            for (int j=0; j
            {
                ret += ((List)columns.get(j)).get(i).toString() + " | ";
            }
            ret += "\n";
        }

        return ret;
 }

I am not surprised to see that the only dependency on the headers is just an empty interface, as a flag.

java 代码

public interface Field
{
}

If you compare the Swing's TableModel implementations to this one, it's almost the same, except the event handling portion, plus we have several extra methods. This is a good sign that we abstract the responsibility at the right level, not too many responsibilities.

The only relevant missing piece is the serialization. We can either implements Serializable or Externalizable.

In this implementation, I delegate the dynamic memory allocation to Collection classes. I could use a Map that maps fields to columns, but I instead use two Lists for speeding.

I also supplied a toString() method for debugging purpose, but left out other formatters, like toXml(), toCvs() etc.

Since this is more of a design issue, I left out most of the code untested. For simple testing, I created a simple field class:

java 代码

public class StringField implements Field
{
    private String name;

    public StringField(String name)
    {
        this.name = name;
    }

    public String getName() { return name; }

    public boolean equals(Object obj)
    {
        if (!(obj instanceof StringField)) return false;

        StringField field = (StringField)obj;
        return this.name.equals(field.name);
    }

    public int hashCode() { return name.hashCode(); }

    public String toString() { return getName(); }
}

A real world Field class would have dependencies on date, currency and other factors.