Phil 3.30.17

7:00 – 8:00, 4:00 – 6:00

  • Looking more closely at Qt and PyQt. First, the integrated browser is nice. Second, if it’s possible to wireframe UIs in Qt and connect them to Python for matrix calculations and server interactions, then I have some real leverage.
  • Really good overview. Difference between 4 and 5, etc.
  • Lotsa python and machine learning videos. These look promising
  • Meeting with Don

8:30 – 3:30 BRC

  • Cleaning up reading code and adding argument handling
  • Need to add a row-reader to replace the slow_read_pbf and the slow_write_pbf methods. They also need to turn the row into a JSON object, and to have a separate method that will assemble a DataFrame from a set of JSON objects in memory
  • Nope, strike that. We’re now going to read CSV files containing sparse data and assemble them into a DataFrame. The format is:
    "member:123456",CNR:3,CPD:9,TAS:1,AFB:38,FUD:1
    "member:789012",PRK:1,TAS:2
  • The row id is quoted, otherwise key:value pairs are comma separated. Rows are terminated by CR/LF, and there can be multiple rows per file.
  • Python hadoop: Pydoop
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: