Monthly Archives: September 2009
7:30 – 3:30
- Deployed the new version – there are a few bugs:
- Although remote files validate, they don’t read in – looks like the problem was that the remote files don’t terminate in .xml
- Blanks are counted in the UniqueCounts view – fixed
- Need to clear out the “where clauses” if the last item is unchecked – fixed
- Need to keep the where clauses in the unique count query construction – fixed
- Had a good discussion with Mike C about data
- Scheduled a meeting with Anne to go over the new features
- Burned a new disc and am ready to deploy tomorrow.
7:30 – 3:3o
- Integrating Dong’s code in prep for release – done
- Burned a disc
Week of 09/28
- IE chokes when URL (endpoint) is not set up with full path; ***.***.com or IP address
- Fixed tables returning from server not role/user based
- Added publish status, changed mode to Overwrite to IngestConfirm window
- Changed date in the table names to return YYYY-MM-DD format
- Bug found in AutoIngestor
- found that files with no ext (.xml, .xls, etc.) fails to return columns
- input validation added to RemoteFileWindow and RoleSelectinCanvas
- Added dataTips to AutoIngest DataGrid
- Rearranged New and Remote File buttons in AutoIngestConfig
- Extending File class to HTTPFile to use remote files
- Adding HTTPS and HTTP capabilities to other types of files
- All changes uploaded and downloaded.
Week of 09/21
- Administrators can now delete any tables from the IngestManager
- isUserAdmin() methods added to both client and server
- Changed displaying ID of the user to Names of the Users in IngestPanel and IngestedDataConfiguration
- Database Query to join TABLE_USERS and TABLE_DATA tables
- StyleFunction for the AdvancedDataGrid
- AutoIngest now checks for existing data (file name and table name) on the client
- Remote File Ingest to handle http and https
- Server – HttpUtillities.java and SSLUtilities.java under com.edgeti.utils
- Flex – added Remote File button in IngestVBox, RemoteFileWindow.mxml to handle remote file ingest
- remote file added to Files Available after validation
- change handleFileSelected() in IngestVBox to handle http and https and related events/services
- columns returned from http and https
- removed old remote file routines from AutoIngestConfigVBox and put RemoteFileWindow to use http and https autoingest…..
- changed server codes to use full filename for remote (http) and uploadDir for local files.
- ingest works!!!! seems nothing is broken….
Week of 09/14
- disabled Use Remote location for Auto Ingest for now….
- fixed auto ingest to run specified time, hourly, daily, weekly.
- creating RoleColumns/RoleTree component so it can be used by IngestVBox and AutoIngest.
Week of 09/07
- Resolved most of the bugs/fixes/enhancements Phil asked.
- Reworking Ingestor, AutoIngest Config, and Ingested Data Config to use Roles and Role Columns. This requires changes to the Servlet, too.
- Ingestor problem resolved
- moved connect/disconnect for each insert SQL to east ingest. It seems ingestion is a lot faster now. XLS ingestor should be changed
- Continued troubleshooting ingestor problem
- Ingestor running too fast?
- file too large?
- Remote File ingestion
- running Tomcat under SYSTEM prevents the Ingestor to read remote files
- running different user (me) works fine.
Week of 08/31
- Test Deploy to EdgeDemo – successful
- change endpoint in assets/setupModel.xml
- MakeFakeData in ManagementUtilsTest.html fails. It seems there is a problem in populateFakeUsers in UsersAndRoles.java
- Having problems with ingestor. MySQL reports socket not available error
- Continue looking into remote file ingesttion
7:30 – 3:30
- Gotta install sp2…. Well that was 20 minutes well spent! Wheee.
- Working on multiple select = done. Leaving the single select code in place for a while out of, well, superstition and fear, actually.
- I’m thinking about adding another chart that has the unique counts for each column – that way it would be possible to easily find out how many people we in a query, even if they had multiple skills. – done
- Search now automatically scrolls to the selected item in the datagrid if it finds one
- Saving / loading now once again works with the AdvQueryWidget
- DynamicPanel may now go directly from minimized to maximized and vice versa
- Updated all Layout Managers to handle the new panel feature
7:30 – 3:30
- Another meeting with John, Anne, Brian, and Mike this time
- Based on getting new data (roster2) John is much happier with the tool. Mind you, there have been no changes to the tool.
- Working on getting an “insert column” capability, *then* we might get to multiple selects – done
7:30 – 4:00
- Long and painful meeting with John and Anne. My notes follow:
- Data names are not clear from source
- Should be able to check a box and get a list of names
- Eliminate redundant columns
- Fix column names
- Either the basic tree or the question tree needs to have some checkboxes that represent “where clauses” that would be OR-ed together
- Collapse redundant entries in columns on Flex side – i.e. how many unique names there are. This may be a new chart view
- Working on Question interface – first pass is done. Rolling up to bring in.
- Tomorrow, work on AdvancedQueryWidget.onCreateChartClicked() so that multiple bars can be selected.
7:30 – 5:00
- Ingested data at the fort. Talked to John, who tried to find info with Pat yesterday. I said that the display is only as good as the data, but I don’t think it stuck.
- Working on “Questions” interface
- Modify – done
- Add – done
- Add as Child – done
- Delete – done – turned out to be tricky. You have to use the delete operator.
- Have “Set Query Builder” populate the _questionQuery.text field
- Add a setTableQuestions(String questions, String tableName) method in DbRemoteObject that a) Creates a table_questions table if there is none, and b) loads the xml as a blob in the table row (note: only one set of questions per table) See the dashboard entry for how to handle xml – done
- Add a getTableQuestions(String tableName) method in DbRemoteObject that a) Creates a table_questions table if there is none, and b) either returns the xml if it exists or returns and empty <questions/> xml string. – done
7:30 – 3:30
- Deployed the new version with Mike’s fixes. Everything seems to be working fine
- Talked to Anne about building special-purpose tables from the master table to make it easier to explore the data
- Floated the question idea
- Realized that we could also have all the available strings in a column listed in the (a?) hierachy view, so some level of drilldown could be available…
- Changed all query double quotes to single quotes
- Phone call from Anne – Meeting with here and John on Thursday. Might have a version by then.
- Started on the question tab, mostly because it’s an easier problem to solve that might give us what we want. Go the part where you can click on a tree component “question” that runs a query.
- Need to be able to interactively build the xml
- Need to save the xml in a table-specific entry (only one entry per table holding the xml)
7:30 – 4:00
- Hey! It’s Fall! No wonder it’s dark when I get up these days
- So I come in this morning and find that Mike has fixed half the issues that Anne and John listed. It’s like having pixies do the dishes at night!
- VIS2 Meeting at 10:30
- Writing up support for S2Dash
- Need to make the system more Pat-freindly. Not sure what that means, but I’m leaning towards “Questions” that point to queries