Monthly Archives: July 2015

Phil 7.31.15

7:30 – 2:30 SR

  • AC service today, so I’m working from home.
  • Got server code running at home, now testing on production server.
  • Had to play around with config files, but everything is working now.
  • Working out a way to identify an item for easy searching when there is no (Google) guid. Trying an sha1 hash of the elements used to make the item.
  • Finished addItem()
  • Finished getItem()
  • Finished addItemRating()
  • Finished getItemRatings()
Advertisements

Phil 7.30.15

8:00 – 4:30 SR

  • Still working on getting software for the new dev machine.
  • Added tn_ratings and tn_touches to handle historical behavior. I realized that items shouldn’t have histories in a relational db. Histories should point at items. Sideways thinking.
  • Setting up base and subclasses for the DbIO.
    • baseDbIo
      • improved fail returns
    • userDbIo
    • networkDbIo
      • getUserNetworks
  • Working on item find/add/change/delete PHP functions

Phil 7.29.15

8:00 – 4:30 SR

Phil 7.28.15

8:00 – 5:00 SR

  • Fixing charts – everything done but TST. Did we do that one by hand?
  • Had a thought that the entire financial system could be represented as a network, which could be made to fit into the framework I’m trying to develop. It might let me collapse the project into something small enough for one person to manage, since the database would shrink to only a few (5?) tables…
  • Ordering Software – Visual Studio and IDEA Ultimate
  • A Visual Introduction to Machine Learning
  • Starting to build the server classes that will access the network data
    • A new component doesn’t have have an id_index, so we will know when to create and when to update. Deletion may be tricky, since multiple networks may share an item. We may just want to keep deleted items around anyway, since the fact that they were once attached might mean something…?
    • Class is up and doing raw calls to the database, which, of course is dangerous as heck. Working through how to do bound parameters. Staring with the addUser(), checkUser() and changePassword() methods, since they’re needed and unlikely to change.
    • Discovered the very nice PHP Data Objects (PDO) and this helpful tutorial. I have the checkUser() and rawSqlQuery() methods converted. Need to add some error checking, but very happy.

Phil 7.27.15

8:00 – 2:30 SR

  • I’m thinking about how the edges in a network can have certain characteristics
    • Bandwidth – the capacity of the edge. Can be expressed as used and potential
    • Frequency – how often the edge is used. Bandwidth provides a ceiling on this
    • Richness – I’m not sure how to think about this. In the most basic case, this could be an expression of information content (i.e. an infrequent transmission with lots of content is equivalent to a frequent transmission with low content). But when we layer on context and meaning, all kinds of additional information gets ‘compressed’ into the message – a glance can be nothing or everything. Ideally, the description of the edge should contain some way of accessing that context, and shouldn’t be considered simply a link
  • Along this lines, I found this paper: Lexical chains as representations of context for the detection and correction of malapropisms. Lexical chains are certainly one way of representing context.
  • Another way of thinking about richness is in terms of similarity. If the link is carrying the same payload over and over again, then there isn’t much richness. So can we look at similarity as a way of determining how rich a link is? An Information-Theoretic Definition of Similarity. And here’s looking at news headlines, which might be relevant to short posts or tweets: Similarity for news recommender systems
  • And this reminded me of Princeton’s WordNet, which could be really helpful.
  • Ok, back to the database
    • Based on the above thoughts, I’m adding assoc_type to tn_associations
    • Here’s a query that pulls all the parts together. According to everything I’ve read, joins are probably the most efficient way to do this (size and speed):
      select u.login as `Link Created By`, a.created_on as `Created On`, at.name as `Assoc Type`, si.text as Source, st.name as `Source Type`, ti.text as Target, tt.name as `Target Type`
      from  tn_associations a
      inner join tn_users u on a.user_id=u.uid
      inner join tn_types at on a.assoc_type = at.uid
      inner join tn_items si on a.source_id = si.uid
      inner join tn_types st on si.item_type = st.uid
      inner join tn_items ti on a.target_id = ti.uid
      inner join tn_types tt on ti.item_type = tt.uid;
    • Now I need to load and save a named network from a particular user or any variants up to and including all networks from all users, without redundant nodes/edges…

Phil 7.24.15

8:00 – 4:30 SR

  • Trying to get charts to work
  • Need to run Create Scratch Financial Data first
  • I think 2014 OM ACC is functioning correctly, but probably pointing at the wrong data. The chart is drawing, just need to figure out how to align the months.
  • Working on setting up the database for links as per the main goal. Think I’ve gotten a first pass on the tables. Need to remember how to use joins now. This helped a lot: http://stackoverflow.com/questions/3709560/mysql-join-three-tables

Phil 7.23.15

9:00 – 5:00 SR

  • First, I’m going to check to see if I can pull in and display an entire webpage with ng-sanitize
  • Had to get a cert for my dev machine php install to get curl to be able to pull https content. Here’s the relevant info from the php.net post
Please everyone, stop setting CURLOPT_SSL_VERIFYPEER to false or 0. If your PHP installation doesn't have an up-to-date CA root certificate bundle, download the one at the curl website and save it on your server:

http://curl.haxx.se/docs/caextract.html

Then set a path to it in your php.ini file, e.g. on Windows:

curl.cainfo=c:\php\cacert.pem

Turning off CURLOPT_SSL_VERIFYPEER allows man in the middle (MITM) attacks, which you don't want!
  • Adding the ability to open full pages. It’s now working (not for all pages, will need to finesse that), but I had a few moments where Chrome would NOT LET GO of its cache. Sheesh.
  • Loading the html in the PHP and sending it back as content didn’t work. The trick is to open the page in a frame directly (and save the link) Based on the stackoverflow staring point.
      • Use the $sce service component from ngSanitize and inject in the main module:
        this.appModule.directive('ngFeedPanel', ['$timeout','$rootScope', '$sce', queryDirectivePtr]);
      • It gets incorporated in the directive so:
        public ctor(timeout:ng.ITimeoutService, rootscope:ng.IScope, sce:ng.ISCEService):ng.IDirective {
            this.sceProvider = sce;
            // other stuff goes here
        }
      • That in turn gets called in the html like this:
        
        
  • Lastly, getLink() in the directive looks like:
    scope.getLink = ():void => {
        var mobj:RssControllersModule.IDataResponse = scope.messageObj;
        return this.sceProvider.trustAsResourceUrl(mobj.link);
    };

Phil 7.22.25

8:00 – 5:00 SR

  • Filled out contact info for Steve
  • The stricter trust chain did not work. I had Ronda go back to the looser one.
  • Fixed attraction/repulsion/linkScalar it’s 1.0, 10.0, 10.0 for defaults
  • Ran into a weirdness with <input type=”range> and browsers. Chrome is fine. FF and Chrome have differing default widths and margin/padding. Had to add the following to get all ranges to work similarly:
    .forceRange{
        position: absolute;
        width: 120px;
        right: 10px;
        margin: 0px;
        padding: 5px;
        z-index: 2;
    }
  • Back to figuring out the AlchemyNews API. Blew up the limit again, being careful. I think that the news API, aside from behaving poorly (things like ‘&q.enriched.url.concepts.concept.relevance=0.9’ don’t work). Think I’m going to add more user interaction and less machine learning. Store it all for later page ranking?

Phil 7.21.15

8:00 – 4:30 SR

  • Server is behaving with the stricter trustchain.
  • This is the AlchemyNews REST API Documentation, and the list of fields that can be returned. And Twitter access, BTW.
  • Blew through my limits for the day trying to figure out keywords. Asking for academic license.
  • A good example of how to group query elements: http://alchemyapi.readme.io/docs/sentiment-analysis
  • Need to add sliders for attraction and repulsion (network scalars?). Implemented. Now I need to figure out some good values. I think that we might just have to scale attraction for linked items. It should clean things up and cut down the math a bit.

Phil 7.20.15

8:00 – 2:30 SR

  • Updated the truancy report.
  • Adding multi-connections to network construction
  • Woohoo!seachStructure
  • Then
    • NewsAPI or database.