Phil 3.28.18

7:00 – 5:00 ASRC MKT

    • Aaron found this hyperparameter optimization service: Sigopt
      • Improve ML models 100x faster
      • SigOpt’s API tunes your model’s parameters through state-of-the-art Bayesian optimization.
      • Exponentially faster and more accurate than grid search. Faster, more stable, and easier to use than open source solutions.
      • Extracts additional revenue and performance left on the table by conventional tuning.
    • A Strategy for Ranking Optimization Methods using Multiple Criteria
      • An important component of a suitably automated machine learning process is the automation of the model selection which often contains some optimal selection of hyperparameters. The hyperparameter optimization process is often conducted with a black-box tool, but, because different tools may perform better in different circumstances, automating the machine learning workflow might involve choosing the appropriate optimization method for a given situation. This paper proposes a mechanism for comparing the performance of multiple optimization methods for multiple performance metrics across a range of optimization problems. Using nonparametric statistical tests to convert the metrics recorded for each problem into a partial ranking of optimization methods, results from each problem are then amalgamated through a voting mechanism to generate a final score for each optimization method. Mathematical analysis is provided to motivate decisions within this strategy, and sample results are provided to demonstrate the impact of certain ranking decisions
    • World Models: Can agents learn inside of their own dreams?
      • We explore building generative neural network models of popular reinforcement learning environments[1]. Our world model can be trained quickly in an unsupervised manner to learn a compressed spatial and temporal representation of the environment. By using features extracted from the world model as inputs to an agent, we can train a very compact and simple policy that can solve the required task. We can even train our agent entirely inside of its own hallucinated dream generated by its world model, and transfer this policy back into the actual environment.
    • Tweaked the SingleNeuron spreadsheet
    • This came up again: A new optimizer using particle swarm theory (1995)
      • The optimization of nonlinear functions using particle swarm methodology is described. Implementations of two paradigms are discussed and compared, including a recently developed locally oriented paradigm. Benchmark testing of both paradigms is described, and applications, including neural network training and robot task learning, are proposed. Relationships between particle swarm optimization and both artificial life and evolutionary computation are reviewed.
      • New: Particle swarm optimization for hyper-parameter selection in deep neural networks
    • Working with the CIFAR10 data now. Tradeoff between filters and epochs:
      NB_EPOCH = 10
      NUM_FIRST_FILTERS = int(32/2)
      NUM_MIDDLE_FILTERS = int(64/2)
      OUTPUT_NEURONS = int(512/2)
      Test score: 0.8670728429794311
      Test accuracy: 0.6972
      Elapsed time =  565.9446044602014
      
      NB_EPOCH = 5
      NUM_FIRST_FILTERS = int(32/1)
      NUM_MIDDLE_FILTERS = int(64/1)
      OUTPUT_NEURONS = int(512/1)
      Test score: 0.8821897733688354
      Test accuracy: 0.6849
      Elapsed time =  514.1915690121759
      
      NB_EPOCH = 10
      NUM_FIRST_FILTERS = int(32/1)
      NUM_MIDDLE_FILTERS = int(64/1)
      OUTPUT_NEURONS = int(512/1)
      Test score: 0.7007060846328735
      Test accuracy: 0.765
      Elapsed time =  1017.0974014300725
      
      Augmented imagery
      NB_EPOCH = 10
      NUM_FIRST_FILTERS = int(32/1)
      NUM_MIDDLE_FILTERS = int(64/1)
      OUTPUT_NEURONS = int(512/1)
      Test score: 0.7243581249237061
      Test accuracy: 0.7514
      Elapsed time =  1145.673343808471
      
    • And yet, something is clearly wrong: wrongPNG
    • Maybe try this version? samyzaf.com/ML/cifar10/cifar10.html

 

Advertisements

Phil 3.26.18

But this occasional timidity is characteristic of almost all herding creatures. Though banding together in tens of thousands, the lion-maned buffaloes of the West have fled before a solitary horseman. Witness, too, all human beings, how when herded together in the sheepfold of a theatre’s pit, they will, at the slightest alarm of fire, rush helter-skelter for the outlets, crowding, trampling, jamming, and remorselessly dashing each other to death. Best, therefore, withhold any amazement at the strangely gallied whales before us, for there is no folly of the beasts of the earth which is not infinitely outdone by the madness of men.

—-Moby Dick, The Grand Armada

8:30 – 4:30 ASRC MKT

  • Finished BIC and put the notes on Phlog
  • Exposure to Opposing Views can Increase Political Polarization: Evidence from a Large-Scale Field Experiment on Social Media
    • There is mounting concern that social media sites contribute to political polarization by creating “echo chambers” that insulate people from opposing views about current events. We surveyed a large sample of Democrats and Republicans who visit Twitter at least three times each week about a range of social policy issues. One week later, we randomly assigned respondents to a treatment condition in which they were offered financial incentives to follow a Twitter bot for one month that exposed them to messages produced by elected officials, organizations, and other opinion leaders with opposing political ideologies. Respondents were re-surveyed at the end of the month to measure the effect of this treatment, and at regular intervals throughout the study period to monitor treatment compliance. We find that Republicans who followed a liberal Twitter bot became substantially more conservative post-treatment, and Democrats who followed a conservative Twitter bot became slightly more liberal post-treatment. These findings have important implications for the interdisciplinary literature on political polarization as well as the emerging field of computational social science.
  • More Keras
  • hyperopt is a Python library for optimizing over awkward search spaces with real-valued, discrete, and conditional dimensions.
  • One Hidden Layer:
    training label size =  60000
    test label size =  10000
    60000 train samples
    10000 test samples
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #   
    =================================================================
    dense_1 (Dense)              (None, 128)               100480    
    _________________________________________________________________
    activation_1 (Activation)    (None, 128)               0         
    _________________________________________________________________
    dense_2 (Dense)              (None, 128)               16512     
    _________________________________________________________________
    activation_2 (Activation)    (None, 128)               0         
    _________________________________________________________________
    dense_3 (Dense)              (None, 128)               16512     
    _________________________________________________________________
    activation_3 (Activation)    (None, 128)               0         
    _________________________________________________________________
    dense_4 (Dense)              (None, 10)                1290      
    _________________________________________________________________
    activation_4 (Activation)    (None, 10)                0         
    =================================================================
    Total params: 134,794
    Trainable params: 134,794
    Non-trainable params: 0
  • Two hidden layers:
    training label size =  60000
    test label size =  10000
    60000 train samples
    10000 test samples
    _________________________________________________________________
    Layer (type)                 Output Shape              Param #   
    =================================================================
    dense_1 (Dense)              (None, 128)               100480    
    _________________________________________________________________
    activation_1 (Activation)    (None, 128)               0         
    _________________________________________________________________
    dense_2 (Dense)              (None, 128)               16512     
    _________________________________________________________________
    activation_2 (Activation)    (None, 128)               0         
    _________________________________________________________________
    dense_3 (Dense)              (None, 128)               16512     
    _________________________________________________________________
    activation_3 (Activation)    (None, 128)               0         
    _________________________________________________________________
    dense_4 (Dense)              (None, 10)                1290      
    _________________________________________________________________
    activation_4 (Activation)    (None, 10)                0         
    =================================================================
    Total params: 134,794
    Trainable params: 134,794
    Non-trainable params: 0

Phil 3.23.18

7:00 – 5:00 ASRC MKT

  • Influence of augmented humans in online interactions during voting events
    • Overwhelming empirical evidence has shown that online social dynamics mirrors real-world events. Hence, understanding the mechanisms leading to social contagion in online ecosystems is fundamental for predicting, and even manouvering, human behavior. It has been shown that one of such mechanisms is based on fabricating armies of automated agents that are known as social bots. Using the recent Italian elections as an emblematic case study, here we provide evidence for the existence of a special class of highly influential users, that we name “augmented humans”. They exploit bots for enhancing both their visibility and influence, generating deep information cascades to the same extent of news media and other broadcasters. Augmented humans uniformly infiltrate across the full range of identified clusters of accounts, the latter reflecting political parties and their electoral ranks.
  • Reddit and the Struggle to Detoxify the Internet
    • “Does free speech mean literally anyone can say anything at any time?” Tidwell continued. “Or is it actually more conducive to the free exchange of ideas if we create a platform where women and people of color can say what they want without thousands of people screaming, ‘Fuck you, light yourself on fire, I know where you live’? If your entire answer to that very difficult question is ‘Free speech,’ then, I’m sorry, that tells me that you’re not really paying attention.”
    • This is the difference between discussion and stampede. That seems like it should be statistically detectable.
  • Metabolic Costs of Feeding Predictively Alter the Spatial Distribution of Individuals in Fish Schools
    • We examined individual positioning in groups of swimming fish after feeding
    • Fish that ate most subsequently shifted to more posterior positions within groups
    • Shifts in position were related to the remaining aerobic scope after feeding
    • Feeding-related constraints could affect leadership and group functioning
    • I wonder if this also keeps the hungrier fish at the front, increasing the effectiveness of gradient detections
  • Listening to Invisibilia: The Pattern Problem. There is a section on using machine learning for sociology. Listening to get the author of the ML and Sociology study. Predictions were not accurate. Not published?
  • The Coming Information Totalitarianism in China
    • The real-name system has two purposes. One is the chilling effect, and it works very well on average netizens but not so much on activists. The other and the main purpose is to be able to locate activists and eliminate them from certain information/opinion platforms, in the same way that opinions of dissident intellectuals are completely eradicated from the traditional media.
  • More BIC – Done! Need to assemble notes
    • It is a central component of resolute choice, as presented by McClennen, that (unless new information becomes available) later transient agents recognise the authority of plans made by earlier agents. Being resolute just is recognising that authority (although McClennen’ s arguments for the rationality and psychological feasibility of resoluteness apply only in cases in which the earlier agents’ plans further the common ends of earlier and later agents). This feature of resolute choice is similar to Bacharach’ s analysis of direction, explained in section 5. If the relationship between transient agents is modelled as a sequential game, resolute choice can be thought of as a form of direction, in which the first transient agent plays the role of director; the plan chosen by that agent can be thought of as a message sent by the director to the other agents. To the extent that each later agent is confident that this plan is in the best interests of the continuing person, that confidence derives from the belief that the first agent identified with the person and that she was sufficiently rational and informed to judge which sequence of actions would best serve the person’s objectives. (pg 197)
  • Meeting with celer scientific
  • More TF with Keras. Really good progress