Category Archives: JavaScript Development

Phil 4.25.16

5:30 – 4:00 VTX

  • Saw this on Twitter about visualizing networks with D3
  • Working my way through the JavaFX tutorial. It is a lot like a blend of Flex and a rethought Swing. Nice, actually…
  • Here is the list of stock components
  • Starting with the ope file dialog – done.
  • Yep, there’s a spinner. And here’s dials and knobs
  • And here’s how to do a word cloud.
  • Here’s a TF-IDF implementation in JAVA. Need to build some code that reads in from our ‘negative match’ ‘positive match’ results and start to get some data driven terms
  • Tregex is a utility for matching patterns in trees, based on tree relationships and regular expression matches on nodes (the name is short for “tree regular expressions”). Tregex comes with Tsurgeon, a tree transformation language. Also included from version 2.0 on is a similar package which operates on dependency graphs (class SemanticGraph, calledsemgrex).
  • Semgrex
  • Sprint review
    • Google CSEs
      • Switched over from my personal CSEs to Vistronix CSEs
      • Added VCS rep for CSEs
      • Figured out how to save out and load CSE from XML
      • Added a few more CSEs ONLY_NET, MOBY_DICK
      • Wrote up care and feeding document for Confluence
      • Added blacklists
    • Rating App
      • Re-rigged the JPA classes to be Ontology-agnostic Version 2 of nearly everything)
      • Upped my JQL game to handle SELECT IN WHERE precompiled queries
      • Reading in VA and PA data now
      • Added the creation of a text JSON object that formalizes the rating of a flag
      • Got hooked up to the Talend DB!!!
      • Deployed initial version(s)
      • Added backlink logging using SemRush
    • Future work
      • Developed Excel ingest
      • Still working on PDF and Word ingest
Advertisements

Phil 11.24.15

7:00 – Leave

  • Constraints: Interpreting Line Drawings
    • Successful research:
      • Finds a problem
      • Finds a method that solves the problem
      • Using some principal (That can be generalized)
  • Gave Aaron M. A subversion account and sent him a description of the structure of the project
  • Back to dictionary creation
    • Wire up Extract into Dictionary
      • I think I’m going to do most of this on the server. If I do a select text from tn_view_network_items where network = X, then I can run that text that is already in the DB through the term extractor, which should be the fastest thing I can do.
      • The next fastest thing would be to pull the text from the url (if it exists) and add that to the text pull.
      • Added a getTextFromNetwork() method to NetworkDbObject.
      • The html was getting extracted badly, so I had to add a call to alchemy to return the cleaned text. TODO: in the future add a ‘clean_text’ column to tn_items so this is done on ingestion. I also added
      • Added all the pieces to the rssPull.php file and tested. And integrated with the client. Looks like it takes about 8 seconds to go through my resume, so some offline processing will probably be needed for ACM papers, for example.
    • Wire up Attach Dictionary to Network
      • The current setup is set so that a new item that is read in will associate with the current network dictionary. Need to add a way to have the items that are already in the network to check themselves against the new dictionary.
      • Added class AlchemyDictReflect that will place keywords in the DB. Still need to debug. And don’t forget that the controller will have to reload the network after all thechanges are made.

 

Phil 11.23.15

7:00 – Leave

  • Search: Games, Minimax, and Alpha-Beta
    • Branching factor (B)
    • Search depth (D)
    • Combining the two gives the number of leaf nodes or B^D
    • Branching factor of chess is approximately 14?
  • Dictionaries
    • Wire up Create New Dictionary – done
    • Wire up Extract into Dictionary
      • I think I’m going to do most of this on the server. If I do a select text from tn_view_network_items where network = X, then I can run that text that is already in the DB through the term extractor, which should be the fastest thing I can do.
      • The next fastest thing would be to pull the text from the url (if it exists) and add that to the text pull.
    • Wire up Attach Dictionary to Network
      • The current setup is set so that a new item that is read in will associate with the current network dictionary. Need to add a way to have the items that are already in the network to check themselves against the new dictionary.

Phil 11.20.15

7:00 – Leave

  • Search: Optimal, Branch and Bound, A*
    • Dead Horse principle.
    • Admissible vs. consistency heuristics.
  • Dictionaries
    • Wire up Add – Done. Also found a bug in the PHP code that was screwing up parent return. So that took a while…
    • Wire up Delete – Done. That went much better!
    • Wire up New Dictionary
    • Wire up Extract
    • Wire up Attach

Phil 11.19.15

7:00 – 5:00 Leave

  • Reasoning: Goal Trees and Rule-Based Expert Systems
    • There, now I’m back in order.
    • H. Simon – The complexity of the behavior is max(cplx(prgm), cplx(env))
    • More Genesis – Elaboration graphs
    • Genesis judges similarity in multiple ways: (this presentation, page 25)
      • Using word vectors
      • Using concept vectors: seeing similarities not evident in the words.
    • Genesis aligns similar stories for analogical reasoning (Needleman-Wunch algorithm, which is a way of comparing string similarity using matrices)
  • IRB renewal – ask Wayne
  • In a fit of orderliness, created shortcuts to the cmd windows that the makefiles run in
  • Dictionaries
    • Don’t forget to move the text-extraction calls to the methods that need it and see if that speeds up the others. Done. Much faster. PHP is dumb. Or needs a preloader/compiler
    • Cleaned up the loading and display code a bit. Need to add a tree view (which looks like it can be done in pure CSS), but that can wait.
    • Adding manual entry
      • Finished the form
      • Clear for entry and separate parent is done
      • addition
      • deletion
      • modification (adding parents in particular)
    • Adding text extraction

Phil 11.18.15

7:00 – 5:00 Leave

  • Search: Depth-First, Hill Climbing, Beam
    • Looks like I missed one. Will go back tomorrow.
    • Patrick Winston is the instructor, and he showed a version of Genesis (around 42 minutes in), which maps stories into graphs and then searches through them at different levels of abstraction. It is capable of providing relevant answers to questions. Here’s a white paper. What I think is interesting here is the following:
      • How they describe their hierarchy (revenge, tit-for-tat, etc). This is built, not computed based on what looks to be a rules engine.
      • How text in a story can be looked at as a graph.
      • And that’s without reading the paper!
  • Working on dictionary directive
    • Ramifying the dictionary choice through the controller. It should be set initially when the network loads (done), but should be changed if a new one is selected.
    • There does need to be a second step where the DictPull is run and the text of the items is compared against the selected dictionary and KEYWORDS items are added to the network. This is when the network’s dictionary index should be changed.
    • Downloading the dictionary.
    • Need to upload a new word
  • 12:00 workshop on Grant Writing in ITE 459
  • 4:00 meeting with Dr. Pan.

11.2.15

8:00 – 5:00 SR

  • Al came on board today. Showed him around the system, discovering that the scripting system wasn’t working on the production server. Fixed that, and downloaded a copy of the documentation for him to look at. Also gave him accounts on the integration server for him to poke around.
  • Fixed the Reqonciler bug. Had to insert the modified query directly into the reqonciler table to get around odd quote-escaping issues.
  • Updated Friday’s work from the repo. Updated the database and ran the term extraction and dictionary tests.
  • Working on dictionary access methods.
    • Got AddEntry and Remove Entry working. Also removed the tn_dictionary table and stuck the dictionary_id in the tn_dictionary_entries table.
    • Added cascade entry/modification of parent if it doesn’t exist. Otherwise the indices won’t work.

Phil 10.30.15

8:00 – 4:00 SR

  • Working from home today, waiting for people to show up.
  • Here’s the fix for the Reqonciler issue:
    • Open Reqonciler in your browser.
    • click Post-Processing button to see all queries
    • double click the one that you disabled this morning to edit, Order 2100, update month 1 year 2 to 100% from month 12 year 1
    • add ” AND NOT ISNULL(bc.uid)” at the end of the query without the double quotes. Make sure there is a space before.
    • Save, run, and check the data
  • In the process of getting my home dev environment working again. I swear I should just do this once a week so it’s less stressful.
    • Fixed the Imagick load so that there is a test for the extension and whether the extension is installed correctly.
    • Disabled the world wide web service so that apache could run on port 80
    • Updates all the files in the Apache htdocs directory. Forgot that I had updated the server access methods to take an object.
    • It occurs to me that I can load up the DB directly on the server if I don’t get everything done with the dictionary by Wednesday.
  • Examine AlchemyNLP and see if there is a hierarchy that can be used. Not without a lot of work.
  • Buy and download the fivefilters term extractor and see how to integrate.
    • Ordered. Waiting for confirmation to show up.
    • Installed. Time to see if it’ll work. It looks good, though possibly slow? starting to put together a dictionary class to examine more deeply.
  • Add dictionary Flyout directive
    • Name the dictionary
    • Choose the networks (add/remove from list) ()
    • Input html, text or url
    • Get the clean text and show the machine extracted terms. We could look up potential definitions too – from wordnik. Set up an account and applied for a developer key.
    • Show a list of selected terms with checkboxes
      • Checked items can be deleted or grouped
      • Items can be added by typing into a field
    • Show a list of ‘group items’.  This displays a list of the items who’s index appears in the ‘parent’ field
      • Selecting an item in this list reorders the item list to show the appropriate group first
  • There should also be a select dictionary option on the network flyout

Phil 10.29.15

8:00 – 4:30 SR

  • Sent Dong screenshots of the issue. He’s checking queries and code now.
  • Added simpleTests($dbObj) to each class in AlchemyNLP
  • Added ‘skill’ ‘capability’ and  ‘task’ as parents in the dictionary
  • Add flyout directive to create and assign dictionaries and entries.
  • Set the dictionary to zero in the networkDbIo.addNetwork()  PHP code and add the dict_id to the typescript interface. Done
  • Make sure that an association between a keyword and another item is always from the keyword. Otherwise PageRank won’t calculate correctly. Done.
  • Chain up the dictionary and add parent keywords to the network (parents point to children). That way, for example, all ‘skills’ can be elevated, while all ‘tasks’ can be suppressed. Done
  • Changed keywords to be ‘editable’ so they have adjustable link weights. It does make the keywords in the network editable as well. May need to just add a slider to ITEMS of certain types. Still need to think about this…
  • Next step is to buy and download the fivefilters term extractor and see how to integrate?

Phil 10.26.15

8:00 – 10:00, 12:00 – 3:00 SR

  • Query Builder is not capturing all of the COGNOS data correctly. Specifically for FY15 the second year data is not being added into the queries and for FY14 the 3rd year of data is not being added into the queries. FY16 which is in year 1 is working correctly.
  • I’ve checked the tables in the database and it looks like everything is going in correctly (e.g. the cognos_obligations table has timestamps from today). So I don’t know if there is a problem with ingest or with the FA queries.
  • Working on getting all the pieces working in the EXPLICIT item case. I have an issue where the ‘query’ field is used for a lot of things – I’m definitely going to have to clean that up. Probably the best thing to do is to align the database and the GUI better.
    • When getting the text for the item to show as HTML, I got an angular ‘sanitizer unable to parse’ error that I couldn’t figure out. After trying
      sceProvider.trustAsResourceUrl
    • in the directive without success, I went back (slowly) through the html. It turned out to be the difference between this:
      $htmlStr .= '<img src="assets/'.$this->rssImage.'" height="80"></a></td>';

      and this:

      $htmlStr .= '<img src="assets/'.$this->rssImage.'"height="80"></a></td>';

      Didja see it? It’s the space before “height=80”. Chrome handled it when I was working it out, but angular chokes.