# Phil 2.19.18

7:30 – 4:30 ASRC MKT

• Back to BIC.
•  (page 102)
•  (pg 107)
•  (pg 107)
• Sociality: Coordinating bodies, minds and groups
• Human interaction, as opposed to aggregation, occurs in face-to-face groups. “Sociality theory” proposes that such groups have a nested, hierarchical structure, consisting of a few basic variations, or “core configurations.” These function in the coordination of human behavior, and are repeatedly assembled, generation to generation, in human ontogeny, and in daily life. If face-to-face groups are “the mind’s natural environment,” then we should expect human mental systems to correlate with core configurations. Features of groups that recur across generations could provide a descriptive paradigm for testable and non-intuitive evolutionary hypotheses about social and cognitive processes. This target article sketches three major topics in sociality theory, roughly corresponding to the interests of biologists, psychologists, and social scientists. These are (1) a multiple levels-of-selection view of Darwinism, part group selectionism, part developmental systems theory; (2) structural and psychological features of repeatedly assembled, concretely situated face-to-face coordination; and (3) superordinate, “unsituated” coordination at the level of large-scale societies. Sociality theory predicts a tension, perhaps unresolvable, between the social construction of knowledge, which facilitates coordination within groups, and the negotiation of the habitat, which requires some correspondence with contingencies in specific situations. This tension is relevant to ongoing debates about scientific realism, constructivism, and relativism in the philosophy and sociology of knowledge.
• These definitions seem to span atomic (mother/child, etc), small group (situated, environmental), and societal (unsituated, normative)
• Coordination occurs to the extent that knowledge and practice domains overlap or are complementary. I suggest that values serve as a medium. Humans live in a value-saturated environment; values are known from interactions with people, natural objects, and artifacts
• Dimension reduction
•  I’m starting to think that agents as gradient descent machines within networks is something to look for:
• Individual Strategy Update and Emergence of Cooperation in Social Networks
• In this article, we critically study whether social networks can explain the emergence of cooperative behavior. We carry out an extensive simulation program in which we study the most representative social dilemmas. For the Prisoner’s Dilemma, it turns out that the emergence of cooperation is dependent on the microdynamics. On the other hand, network clustering mostly facilitates global cooperation in the Stag Hunt game, whereas degree heterogeneity promotes cooperation in Snowdrift dilemmas. Thus, social networks do not promote cooperation in general, because the macro-outcome is not robust under change of dynamics. Therefore, having specific applications of interest in mind is crucial to include the appropriate microdetails in a good model.
• Alex Peysakhovich and Adam Lerer
• Prosocial learning agents solve generalized Stag Hunts better than selfish ones
• Deep reinforcement learning has become an important paradigm for constructing agents that can enter complex multi-agent situations and improve their policies through experience. One commonly used technique is reactive training – applying standard RL methods while treating other agents as a part of the learner’s environment. It is known that in general-sum games reactive training can lead groups of agents to converge to inefficient outcomes. We focus on one such class of environments: Stag Hunt games. Here agents either choose a risky cooperative policy (which leads to high payoffs if both choose it but low payoffs to an agent who attempts it alone) or a safe one (which leads to a safe payoff no matter what). We ask how we can change the learning rule of a single agent to improve its outcomes in Stag Hunts that include other reactive learners. We extend existing work on reward-shaping in multi-agent reinforcement learning and show that that making a single agent prosocial, that is, making them care about the rewards of their partners can increase the probability that groups converge to good outcomes. Thus, even if we control a single agent in a group making that agent prosocial can increase our agent’s long-run payoff. We show experimentally that this result carries over to a variety of more complex environments with Stag Hunt-like dynamics including ones where agents must learn from raw input pixels.
• The Good, the Bad, and the Unflinchingly Selfish: Cooperative Decision-Making Can Be Predicted with High Accuracy Using Only Three Behavioral Types
• The human willingness to pay costs to benefit anonymous others is often explained by social preferences: rather than only valuing their own material payoff, people also care in some fashion about the outcomes of others. But how successful is this concept of outcome-based social preferences for actually predicting out-of-sample behavior? We investigate this question by having 1067 human subjects each make 20 cooperation decisions, and using machine learning to predict their last 5 choices based on their first 15. We find that decisions can be predicted with high accuracy by models that include outcome-based features and allow for heterogeneity across individuals in baseline cooperativeness and the weights placed on the outcome-based features (AUC=0.89). It is not necessary, however, to have a fully heterogeneous model — excellent predictive power (AUC=0.88) is achieved by a model that allows three different sets of baseline cooperativeness and feature weights (i.e. three behavioral types), defined based on the participant’s cooperation frequency in the 15 training trials: those who cooperated at least half the time, those who cooperated less than half the time, and those who never cooperated. Finally, we provide evidence that this inclination to cooperate cannot be well proxied by other personality/morality survey measures or demographics, and thus is a natural kind (or “cooperative phenotype”)
• “least”, “intermediate” and “most” cooperative. Doesn’t give percentages, though it says that 17.8% were cooperative?

• Talk Susan Gregurick (susan.gregurick@nih.gov)
• All of Us research program
• Opiod epidemic – trajectory modeling?
• PZM21 computational drug
• Develop advanced software and tools. Specialized generalizable and accessible tools for biomedicing (finding stream). Includes mobile, data indexing, etc.
• NIH Data Fellows? Postdocs to senior industry
• T32 funding? Mike Summers at UMBC
• ncbi-hackathons.github.io (look for data?
• Primary supporter for machine learning is NIMH (imaging), then NIGNS, and NCI Team science (Multi-PI) is a developing thing
• 400m in computing enabled interactions (human in the loop decision tools. Research Browser? • Big Data to Knowledge Initiative (BD2K) datascience.nih.gov/bd2k • Interagency Modeling and Analysis Group (IMAG) imagewiki,nibib.nih.gov • funding: bisti.nih.gov • NIH RePorter projectreporter.nih.gov Check out matchmaker. What’s the ranking algorithm? • NIDDK predictive analytics for budgeting <- A2P-ish? • Most of thi srequires preliminary data and papers to be considered for funding. There is one opportunity for getting funding to get preliminary data. Need to get more specific infor here. • Each SRO normalizes grade as a percentile, not the score, since some places inflate, and others are hard. • Richard Aargon at NIGMS • Office of behavioral and social science – NIH center Francis Collins. Also agent-based simulation • Really wants a Research Browser to go through proposals • Fika – study design • IRB – you can email and chat with the board if you have a tricky study Advertisements # Phil 2.16.18 7:00 – 3:00 ASRC MKT • Finished the first draft of the CI 2018 extended abstract! • And I also figured out how to run the sub projects in the Ultimate Angular src collection. You need to go to the root directory for the chapter, run yarn install, then yarn start. Everything works then. • Trolls on Twitter: How Mainstream and Local News Outlets Were Used to Drive a Polarized News Agenda • This is the kind of data that compels us to rethink how we understand Twitter — and what I feel are more influential platforms for reaching regular people that include Facebook, Instagram, Google, and Tumblr, as well as understand ad tech tracking and RSS feedharvesting as part of the greater propaganda ecosystem. • NELA News credibility classification toolkit • The News Landscape (NELA) Toolkit is an open source toolkit for the systematic exploration of the news landscape. The goal of NELA is to both speed up human fact-checking efforts and increase the understanding of online news as a whole. NELA is made up of multiple indepedent modules, that work at article level granularity: reliability prediction, political impartiality prediction, text objectivity prediction, and reddit community interest prediction. As well as, modules that work at source level granularity: reliability prediction, political impartiality prediction, content-based feature visualization. • New benchmarks for approximate nearest neighbors • I built ANN-benchmarksto address this. It pits a bunch of implementations (including Annoy) against each other in a death match: which one can return the most accurate nearest neighbors in the fastest time possible. It’s not a new project, but I haven’t actively worked on it for a while. • Systems of Global Governance in the Era of Human-Machine Convergence • Technology is increasingly shaping our social structures and is becoming a driving force in altering human biology. Besides, human activities already proved to have a significant impact on the Earth system which in turn generates complex feedback loops between social and ecological systems. Furthermore, since our species evolved relatively fast from small groups of hunter-gatherers to large and technology-intensive urban agglomerations, it is not a surprise that the major institutions of human society are no longer fit to cope with the present complexity. In this note we draw foundational parallelisms between neurophysiological systems and ICT-enabled social systems, discussing how frameworks rooted in biology and physics could provide heuristic value in the design of evolutionary systems relevant to politics and economics. In this regard we highlight how the governance of emerging technology (i.e. nanotechnology, biotechnology, information technology, and cognitive science), and the one of climate change both presently confront us with a number of connected challenges. In particular: historically high level of inequality; the co-existence of growing multipolar cultural systems in an unprecedentedly connected world; the unlikely reaching of the institutional agreements required to deviate abnormal trajectories of development. We argue that wise general solutions to such interrelated issues should embed the deep understanding of how to elicit mutual incentives in the socio-economic subsystems of Earth system in order to jointly concur to a global utility function (e.g. avoiding the reach of planetary boundaries and widespread social unrest). We leave some open questions on how techno-social systems can effectively learn and adapt with respect to our understanding of geopolitical complexity. # Phil 2.13.18 7:00 – 4:00 ASRC MKT • UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction • UMAP (Uniform Manifold Approximation and Projection) is a novel manifold learning technique for dimension reduction. UMAP is constructed from a theoretical framework based in Riemannian geometry and algebraic topology. The result is a practical scalable algorithm that applies to real world data. The UMAP algorithm is competitive with t-SNE for visualization quality, and arguably preserves more of the global structure with superior run time performance. Furthermore, UMAP as described has no computational restrictions on embedding dimension, making it viable as a general purpose dimension reduction technique for machine learning. • How Prevalent are Filter Bubbles and Echo Chambers on Social Media? Not as Much as Conventional Wisdom Has It • Yet, as Rasmus points out, conventional wisdom seems to be stuck with the idea that social media constitute filter bubbles and echo chambers, where most people only, or mostly, see political content they already agree with. It is definitely true that there is a lot of easily accessible, clearly identifiable, highly partisan content on social media. It is also true that, to some extent, social media users can make choices as to which sources they follow and engage with. Whether people use these choice affordances solely to flock to content reinforcing their political preferences and prejudices, filtering out or avoiding content that espouses other viewpoints, is, however, an empirical question—not a destiny inscribed in the way social media and their algorithms function. • He Predicted The 2016 Fake News Crisis. Now He’s Worried About An Information Apocalypse. • That future, according to Ovadya, will arrive with a slew of slick, easy-to-use, and eventually seamless technological tools for manipulating perception and falsifying reality, for which terms have already been coined — “reality apathy,” “automated laser phishing,” and “human puppets.” • Finish first pass at DC slides – done! • Begin trimming paper – good progress. • Add a slider that lets the user interactively move a token along the selected trajectory path – done. Yes, it looks like a golf ball on a tee… • Sprint planning # Phil 2.12.18 7:00 – 4:00 ASRC MKT • The social structural foundations of adaptation and transformation in social–ecological systems • Social networks are frequently cited as vital for facilitating successful adaptation and transformation in linked social–ecological systems to overcome pressing resource management challenges. Yet confusion remains over the precise nature of adaptation vs. transformation and the specific social network structures that facilitate these processes. Here, we adopt a network perspective to theorize a continuum of structural capacities in social–ecological systems that set the stage for effective adaptation and transformation. We begin by drawing on the resilience literature and the multilayered action situation to link processes of change in social–ecological systems to decision making across multiple layers of rules underpinning societal organization. We then present a framework that hypothesizes seven specific social–ecological network configurations that lay the structural foundation necessary for facilitating adaptation and transformation, given the type and magnitude of human action required. A key contribution of the framework is explicit consideration of how social networks relate to ecological structures and the particular environmental problem at hand. Of the seven configurations identified, three are linked to capacities conducive to adaptation and three to transformation, and one is hypothesized to be important for facilitating both processes. • Starting to trim paper down to three pages • Starting on CHIIR slide stack – Still need to add future work • Springt Review • Rwanda radio transcripts • From October 1993 to late 1994, RTLM was used by Hutu leaders to advance an extremist Hutu message and anti-Tutsi disinformation, spreading fear of a Tutsi genocide against Hutu, identifying specific Tutsi targets or areas where they could be found, and encouraging the progress of the genocide. In April 1994, Radio Rwanda began to advance a similar message, speaking for the national authorities, issuing directives on how and where to kill Tutsis, and congratulating those who had already taken part. • Fika • Set up Fika Writing group that will meet Wednesdays at 4:00. We’ll see how that goes. # Phil 2.1.18 7:00 – 3:30 ASRC MKT • Communications Handbook for IPCC scientists • The Barnes-Hut Approximation • Efficient computation of N-body forces By: Jeffrey Heer Computers can serve as exciting tools for discovery, with which we can model and explore complex phenomena. For example, to test theories about the formation of the universe, we can perform simulations to predict how galaxies evolve. To do this, we could gather the estimated mass and location of stars and then model their gravitational interactions over time. • Need to get started on the extended abstract for Collective Intelligence 2018! One month! March 2, 2018! • Set up the LaTex template for the conference. Done • Think I want to call it Mapping Simon’s Anthill • Need to contact the CHIIR 2018 folks to see what is expected for the DC • More Angular, feeling my way through the Http code, which has been deprecated. Looked at the similar code in Tour of Heroes. We’ll see if the old stuff works and then try to update? Need to ask Jeremy. • Back to BIC. Evolutionary reasons for cooperation as group fitness, where group payoff is maximized. This makes the stag salient in stag hunt. • A thorough explanation of synchronization/phase locking. My mental model is this: Imaging a set of coaxial but randomly oscillating identical weights sliding back and forth in their section of lightweight tubing. From the outside, the tube would be stationary, as all the forces would be cancelling. If the weights can synchronize, then the lightweight tube will be doing most of the moving. Since the mass of the tube is lower than the mass of the combined weights, The force required for the whole system will be lower, and as a result (I think?) the system will run more efficiently and longer. Need to work out the math. # Phil 1.31.18 7:00 – 7:00 ASRC MKT • The Matrix Calculus You Need For Deep Learning • Most of us last saw calculus in school, but derivatives are a critical part of machine learning, particularly deep neural networks, which are trained by optimizing a loss function. Pick up a machine learning paper or the documentation of a library such as PyTorch and calculus comes screeching back into your life like distant relatives around the holidays. And it’s not just any old scalar calculus that pops up—you need differential matrix calculus, the shotgun wedding of linear algebra and multivariate calculus. • Continuing BIC • Explaining the evolution of any human behavior trait (say, a tendency to play C in Prisoner’s Dilemmas) raises three questions. The first is the behavior selection question: why did this trait, rather than some other, get selected by natural selection? Answering this involves giving details of the selection process, and saying what made the disposition confer fitness in the ecology in which selection took place. But now note that ‘When a behavior evolves, a proximate mechanism also must evolve that allows the organism to produce the target behavior. Ivy plants grow toward the light. This is a behavior, broadly construed. For phototropism to evolve, there must be some mechanism inside of ivy plants that causes them to grow in one direction rather than in another’ (Sober and Wilson 1998, pp. 199-200). This raises the second question, the production question: how is the behavior produced within the individual-what is the ‘proximate mechanism’? In the human case, the interest is often in a psychological mechanism: we ask what perceptual, affective and cognitive processes issue in the behavior. Finally, note that these processes must also have evolved, so an answer to the second question brings a third: why did this proximate mechanism evolve rather than some other that could have produced the same behavior? This is the mechanism selection question. (pg 95) • These are good questions to answer, or at least address. Roughly, I thing my answers are • Selection Question: The three phases are a very efficient way to exploit an environment • Production Question: Neural coupling, as developed in physical swarms and moving on to cognitive clustering • Mechanism Question: Oscillator frequency locking provides a natural foundation for collective behavior. Dimension reduction is how axis are selected for matching. • Value Orientations, Expectations and Voluntary Contributions in Public Goods • Discussion with Aaron about JuryRoom design • Observable is a better way to code. • Discover insights faster and communicate more effectively with interactive notebooks for data analysis, visualization, and exploration. • More Angular. Finished with module communication, starting with services • Meeting with Wayne • Submit to JASS • Abstract to CI 2018 July 7-8, 2018 at the University of Zurich, Switzerland # Phil 1.30.18 7:00 – 5:00 ASRC MKT • Big thought for today.In a civilization context, the three phases of collective intelligence work like this. These phases relate to computational effort which is proportional to the number of dimensions that an individual has to consider in their existential calculus. The assumption is that lower computational effort is selected for at natural explore/exploit ratios. • Exploration phase. Nomadic explorers are introduced to a new environment. Can be physical, informational, cognitive, etc. This phase has the highest dimensional processing required for the individual. • Exploitation phase. Social patterns increase the hill climbing power of agents in the environment. This results in a sufficiently optimal access to resources. This employs lower dimensions to support consensus and polarization. • Inertial phase. Social influence becomes dominant and environmental influence wains. Local diversity drops as similar agents cluster tightly together. Resources wane. This employs the most dimension reduction and the highest polarization, resulting in high implicit coordination. • Collapse. Implied, since the Inertial phase is unsustainable. If the previous population produced explorers that found new, productive environments, the cycle can repeat elsewhere. • Continuing BIC • “We need to know, in detail, what deliberations are like that people engage in when they group-identify”. Also, agency transformation • Rules, norms and institutional erosion: Of non-compliance, enforcement and lack of rule of law • What I am seeing right now in the US (a steady and slow erosion of democratic norms and a systematic violation of rules by the President Elect, in particular as though “they don’t apply to him“) is something that I’ve seen in other countries where I have studied formal and informal rules and institution building (and decay). This, in my view, is worrisome. If the US is going to want to continue having a functioning democracy where compliance with rules and norms is an expectation at the societal level, it’s going to have to do something major to stop this systematic rule violation. • Evaluation of Interactive Machine Learning Systems • The evaluation of interactive machine learning systems remains a difficult task. These systems learn from and adapt to the human, but at the same time, the human receives feedback and adapts to the system. Getting a clear understanding of these subtle mechanisms of co-operation and co-adaptation is challenging. In this chapter, we report on our experience in designing and evaluating various interactive machine learning applications from different domains. We argue for coupling two types of validation: algorithm-centered analysis, to study the computational behaviour of the system; and human-centered evaluation, to observe the utility and effectiveness of the application for end-users. We use a visual analytics application for guided search, built using an interactive evolutionary approach, as an exemplar of our work. We argue that human-centered design and evaluation complement algorithmic analysis, and can play an important role in addressing the “black-box” effect of machine learning. Finally, we discuss research opportunities that require human-computer interaction methodologies, in order to support both the visible and hidden roles that humans play in interactive machine learning. • Jensen–Shannon divergence – I think I can use this to show the distance between a full coordination matrix and one that contains only the main diagonal. • Evolution of social behavior in finite populations: A payoff transformation in general n-player games and its implications • The evolution of social behavior has been the focus of many theoretical investigations, which typically have assumed infinite populations and specific payoff structures. This paper explores the evolution of social behavior in a finite population using a general n-player game. First, we classify social behaviors in a group of n individuals based on their effects on the actor’s and the social partner’s payoffs, showing that in general such classification is possible only for a given composition of strategies in the group. Second, we introduce a novel transformation of payoffs in the general n-player game to formulate explicitly the effects of a social behavior on the actor’s and the social partners’ payoffs. Third, using the transformed payoffs, we derive the conditions for a social behavior to be favored by natural selection in a well-mixed population and in the presence of multilevel selection. • Got the data for the verdicts and live verdicts set up right, or at least closer: • Booked a room for the CHIIR Hotel • Got farther on UltimateAngular: • # Phil 1.29.18 7:00 – 5:30 ASRC MKT • The phrase “Epistemic Game Theory” occurred to me in the shower. Looked it up and found these two things: • When it’s easier to agree than discuss, it should be easier to stampede: • This is also a piece of Salganik’s work as described in Leading the Herd Astray: An Experimental Study of Self-Fulfilling Prophecies in an Artificial Cultural Market • An article on FB optimization and how to change the ratio of likes to comments, etc • I don’t think people did. It’s just that it’s easier to not think too much 🙂 people are busy selling tools that do everything for people, and people are happy buying tools to limit thinking. The analogy of replacing cognitive load with perception by VIS misleads in this regard. (Twitter) • Continuing BIC • Dimension reduction is a form of induced conceptual myopia (pg 89)? • AI Roundup workshop today • Zenpeng, Biruh, Phil, Aaron, Eric, Eric, Kevin • Eric – Introductory remarks. Budget looks good for 2018. Direction, chance to overlap, get leaders together for unique differentiators and something that we can build a business around. There has to be a really good business case with revenue in the out years • Aaron – CDS for A2P. Collaborate on analytics, ML, etc. Non corporate focused. Emerging technologies and trends. Helping each other out. Background in IC software dev. • Pam Scheller – SW Aegis. BD. EE, MS Computer engineering. • Biruh, TF, LIDAR, Generalized AI as hobby. • Zhenpeng Lee – Physics, Instrument Data Processing for GOES-R. FFT. GOES_R radiometric analysis. 7k detector rows? Enormous data sets. Attempting to automate processing the analysis of these data sets. Masters in Computer Science from JHU. Written most of his code from scratch. • Kevin Wainwright. Software engineering Aegis. C&C, etc. Currently working on a cloud based analytics with ML for big data, anomaly detection, etc. Looking for deviation from known flight paths • Eric Velte. History degree. Aegis. Situational awareness. Chief technologists for missions solutions group. Software mostly. Data analytics for the last two years. Big Data Analytics Platform. • Cornel as engineer, Zero G heat transfer, spacecraft work. Technology roadmaps for thermal control. Then business development, mostly for DoD. Research Sports research – head of Olympic Committee research kayaks, women’s 8, horse cooling, bobsleds. • Mike Beduck. Chemical Engineering and computer science. Visualization, new to big data. Closed system sensor fusion. RFP response, best practices. Repository for analytics • George. Laser physics. Cardiac imaging analysis. Software development, 3D graphics. Medical informatics. CASI ground systems. More GOES-R/S. Image and signal processing and analysis. • Anton is lurling and listening. Branding and marketing. • A2P WIP • Put a place on sharepoint for papers and other documents – annotated bibliography. • Floated the JuryRoom app. Need to mention that the polarizing discussion closes at consensus. • Zhenpeng Lee AIMS – GOES-R. What went wrong and how to fix. ML to find pattern change in 20k sensor streams. Full training on each day’s data, then large scale clustering. Trends are seasonal? Relationships between sensors? Channel has 200-600 detectors. “Machine Learning of Situational Awareness” MLP written in Java. TANH activation function. • Eric Haught: Long term quest for condition-based maintenance. • Aaron – we are all trying to come up with a useful cross platform approach to anomaly detection. • Training size: 100k samples? Sample selection reduce to 200? Not sure what the threshold sensitivity is • Eric Velte – Devops. Centralize SW dev and support into a standardized framework. NO SECURITY STACK!!!!! • Dataforbio? Video series # Phil 1.26.18 7:00 – 4:00 ASRC MKT • Tweaked my hypotheses from this post. I need to promote to a Phlog page. • Using Self-Organizing Maps to solve the Traveling Salesman Problem • The Traveling Salesman Problem is a well known challenge in Computer Science: it consists on finding the shortest route possible that traverses all cities in a given map only once. To solve it, we can try to apply a modification of the Self-Organizing Map (SOM) technique. Let us take a look at what this technique consists, and then apply it to the TSP once we understand it better. • Starting JuryRoom project with Jeremy. • Angular material design • VerdictBox (Scenario and verdict) • Chat message • Live discussion cards (right gutter) • Topics (alphabetic, ranking, trending) with sparklines • Progress!!!!!! # Phil 1.23.18 7:00 – 5:00 ASRC MKT • Lesser-known trolley problem variations • News presented as a list: The 270 people connected to the Russia probes • continuing BIC • Groups are defined by a common location, orientation, and velocity through a physical or virtual space. They influence each other dependent on awareness and trust. The lower the number of dimensions, the easier it is to produce a group. • Russia’s Full Spectrum Propaganda • This post examines one full spectrum case to illustrate the method. @DFRLab examined this case in an earlier post; since then, further evidence emerged, which changed and improved our understanding of the technique. • More Angular. Nice progress. I had some issues where I wanted to keep an old version of the app directory and did a refactor. This (of course) refactored the calling program, so I broke quite a few things figuring it out. That being said, Angular 1.5 is really, really nice. • Long chat about handling Trolls in the discussion app # Phil 8.18.17 7:00 – 8:00 Research • Got indexFromLocation() working. It took some fooling around with Excel. Here’s the method: public int[] indexFromLocation(double[] loc){ int[] index = new int[loc.length]; for(int i = 0; i < loc.length; ++i){ double findex = loc[i]/mappingStep; double roundDown = Math.floor(findex); double roundUp = Math.ceil(findex); double lowdiff = findex - roundDown; double highdiff = roundUp - findex; if(lowdiff < highdiff){ index[i] = (int)roundDown; }else{ index[i] = (int)roundUp; } } return index; } • And here are the much cleaner results: • [0.00, 0.00] = [0, 0] [0.00, 0.10] = [0, 0] [0.00, 0.20] = [0, 1] [0.00, 0.30] = [0, 1] [0.00, 0.40] = [0, 2] [0.00, 0.50] = [0, 2] [0.00, 0.60] = [0, 2] [0.00, 0.70] = [0, 3] [0.00, 0.80] = [0, 3] [0.00, 0.90] = [0, 4] [0.00, 1.00] = [0, 4] [1.00, 0.00] = [4, 0] [1.00, 0.10] = [4, 0] [1.00, 0.20] = [4, 1] [1.00, 0.30] = [4, 1] [1.00, 0.40] = [4, 2] [1.00, 0.50] = [4, 2] [1.00, 0.60] = [4, 2] [1.00, 0.70] = [4, 3] [1.00, 0.80] = [4, 3] [1.00, 0.90] = [4, 4] [1.00, 1.00] = [4, 4] • Another thought that struck me as far as the (int) constraint is that I can have a number of ArrayLists that are embedded in a an object that has the first and last index in it. These would be linked together to provide unconstrained (MAX_VALUE or 2,147,483,647 lists) storage 8:30 – 4:30 BRI • I realized yesterday that the Ingest and Query microservices need to access the same GeoMesa Spring service. That keeps all the general store/query GeoMesa access code in one place, simplifies testing and allows for DI to provide the correct (hbase, accumulo, etc) implementation through a facade interface. • Got tangled up with getting classpaths right and importing the proper libraries • Got the maven files behaving, or at least not complaining on mvn clean and mvn compile! • Well that’s a new error: Error: Could not create the Java Virtual Machine. I get that running the new installation with the geomesa-quickstart-hbase • Ah, that’s what will happen when you paste your command-line arguments into the VM arguments space just above where it should go… • Wednesday’s goal will to verify that HBaseQuickStart is running correctly in its new home and start to turn it into a service. # Phil 8.17.17 BRI – one hour chasing down research hours from Jan – May 7:00 – 6:00 Research • Found this on negative flocking influences: The rise of negative partisanship and the nationalization of US elections in the 21st century. Paper saved to Lit Review • One of the most important developments affecting electoral competition in the United States has been the increasingly partisan behavior of the American electorate. Yet more voters than ever claim to be independents. We argue that the explanation for these seemingly contradictory trends is the rise of negative partisanship. Using data from the American National Election Studies, we show that as partisan identities have become more closely aligned with social, cultural and ideological divisions in American society, party supporters including leaning independents have developed increasingly negative feelings about the opposing party and its candidates. This has led to dramatic increases in party loyalty and straight-ticket voting, a steep decline in the advantage of incumbency and growing consistency between the results of presidential elections and the results of House, Senate and even state legislative elections. The rise of negative partisanship has had profound consequences for electoral competition, democratic representation and governance. • Working on putting together an indexable high-dimension matrix that can contain objects. Generally, I’d expect it to be doubles, but I can see Strings and Objects as well. • Starting off by seeing what’s in the newest Apache Commons Math (v 3.6.1) • Found SimpleTensor, which uses the Efficient Java Matrix Library (EJML) and creates a 3D block of rows, columns and slices. THought it was what I wanted, but nope • Looks like there isn’t a class that would do what I need to do, or that I can even modify. I’m thinking that the best option is to use org.apache.commons.math3.linear.AbstractRealMatrix as a template. • Nope, coudn’t figure out how to do things as nested lists. So I’m doing it C-Style, where you really only have one array that you index into. Here’s a 4x4x4x4 Tensor filled with zeroes: Total elements = 256 0.0:[0, 0, 0, 0], 0.0:[1, 0, 0, 0], 0.0:[2, 0, 0, 0], 0.0:[3, 0, 0, 0], 0.0:[0, 1, 0, 0], 0.0:[1, 1, 0, 0], 0.0:[2, 1, 0, 0], 0.0:[3, 1, 0, 0], 0.0:[0, 2, 0, 0], 0.0:[1, 2, 0, 0], 0.0:[2, 2, 0, 0], 0.0:[3, 2, 0, 0], 0.0:[0, 3, 0, 0], 0.0:[1, 3, 0, 0], 0.0:[2, 3, 0, 0], 0.0:[3, 3, 0, 0], 0.0:[0, 0, 1, 0], 0.0:[1, 0, 1, 0], 0.0:[2, 0, 1, 0], 0.0:[3, 0, 1, 0], …. 0.0:[0, 2, 3, 3], 0.0:[1, 2, 3, 3], 0.0:[2, 2, 3, 3], 0.0:[3, 2, 3, 3], 0.0:[0, 3, 3, 3], 0.0:[1, 3, 3, 3], 0.0:[2, 3, 3, 3], 0.0:[3, 3, 3, 3] • The only issue that I currently have is that ArrayLists are indexed by int, so the total size is 32k elements. That should be good enough for now, but it will need to be fixed. • set() and get() work nicely: lt.set(new int[]{0, 1, 0, 0}, 9.9); lt.set(new int[]{3, 3, 3, 3}, 3.3); System.out.println("[0, 1, 0, 0] = " + lt.get(new int[]{0, 1, 0, 0})); System.out.println("[3, 3, 3, 3] = " + lt.get(new int[]{3, 3, 3, 3})); [0, 1, 0, 0] = 9.9 [3, 3, 3, 3] = 3.3 • Started the indexFromLocation method, but this is too sloppy: index[i] = (int)Math.floor(Math.round(loc[i]/mappingStep)); # Phil 8.16.17 7:00 – 8:00 Research • Added takeaway thoughts to my C&C writeup. • Working out how to add capability to the sim for P&RCH paper. My thoughts from vacation: • The agents contribution is the heading and speed • The UI is what the agent’s can ‘see’ • The IR is what is available to be seen • An additional part might be to add the ability to store data in the space. Then the behavior of the IR (e.g. empty areas) would b more apparent, as would the effects of UI (only certain data is visible, or maybe only nearby data is visible) Data could be a vector field in Hilbert space, and visualized as color. • Updated IntelliJ • Working out how to to have a voxel space for the agents to move through that can also be drawn. It’s any number of dimensions, but it has to project to 2D. In the case of the agents, I just choose the first two axis. Each agent has an array of statements that are assembled into a belief vector. The space can be an array of beliefs. Are these just constructed so that they fill a space according to a set of rules? Then the xDimensionName and yDimensionName axis would go from (0, 1), which would scale to stage size? IR would still be a matter of comparing the space to the agent’s vector. Hmm. • This looks really good from an information horizon perspective: The Role of the Information Environment in Partisan Voting • Voters are often highly dependent on partisanship to structure their preferences toward political candidates and policy proposals. What conditions enable partisan cues to “dominate” public opinion? Here I theorize that variation in voters’ reliance on partisanship results, in part, from the opportunities their environment provides to learn about politics. A conjoint experiment and an observational study of voting in congressional elections both support the expectation that more detailed information environments reduce the role of partisanship in candidate choice 9:00 – 5:00 BRI • Good lord, the BoA corporate card comes with SIX seperate documents to read. • Onward to Chapter Three and Spring database interaction • Well that’s pretty clean. I do like the JdbcTemplate behaviors. Not sure I like the way you specify the values passed to the query, but I can’t think of anything better if you have more than one argument: @Repository public class EmployeeDaoImpl implements EmployeeDao { @Autowired private DataSource dataSource; @Autowired private JdbcTemplate jdbcTemplate; private RowMapper<Employee> employeeRowMapper = new RowMapper<Employee>() { @Override public Employee mapRow(ResultSet rs, int i) throws SQLException { Employee employee = new EmployeeImpl(); employee.setEmployeeAge(rs.getInt("Age")); employee.setEmployeeId(rs.getInt("ID")); employee.setEmployeeName(rs.getString("FirstName") + " " + rs.getString("LastName")); return employee; } }; @Override public Employee getEmployeeById(int id) { Employee employee = null; employee = jdbcTemplate.queryForObject( "select * from Employee where id = ?", new Object[]{id}, employeeRowMapper ); return employee; } public List<Employee> getAllEmployees() { List<Employee> eList = jdbcTemplate.query( "select * from Employee", employeeRowMapper ); return eList; } } • Here’s the xml to wire the thing up: <context:component-scan base-package="org.springframework.chapter3.dao"/> <bean id="employeeDao" class="org.springframework.chapter3.dao.EmployeeDaoImpl"/> <bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource"> <property name="driverClassName" value="{jdbc.driverClassName}" />
<property name="url" value="\${jdbc.url}" />
</bean>

<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSource" />
</bean>

<context:property-placeholder location="jdbc.properties" />
• And here’s the properties. Note that I had to disable SSL:
jdbc.driverClassName=com.mysql.jdbc.Driver
jdbc.url=jdbc:mysql://localhost:3306/sandbox?autoReconnect=true&useSSL=false

# Phil 4.25.16

5:30 – 4:00 VTX

• Working my way through the JavaFX tutorial. It is a lot like a blend of Flex and a rethought Swing. Nice, actually…
• Here is the list of stock components
• Starting with the ope file dialog – done.
• Yep, there’s a spinner. And here’s dials and knobs
• And here’s how to do a word cloud.
• Here’s a TF-IDF implementation in JAVA. Need to build some code that reads in from our ‘negative match’ ‘positive match’ results and start to get some data driven terms
• Tregex is a utility for matching patterns in trees, based on tree relationships and regular expression matches on nodes (the name is short for “tree regular expressions”). Tregex comes with Tsurgeon, a tree transformation language. Also included from version 2.0 on is a similar package which operates on dependency graphs (class SemanticGraph, calledsemgrex).
• Semgrex
• Sprint review
• Switched over from my personal CSEs to Vistronix CSEs
• Added VCS rep for CSEs
• Figured out how to save out and load CSE from XML
• Added a few more CSEs ONLY_NET, MOBY_DICK
• Wrote up care and feeding document for Confluence
• Rating App
• Re-rigged the JPA classes to be Ontology-agnostic Version 2 of nearly everything)
• Upped my JQL game to handle SELECT IN WHERE precompiled queries
• Reading in VA and PA data now
• Added the creation of a text JSON object that formalizes the rating of a flag
• Got hooked up to the Talend DB!!!
• Deployed initial version(s)
• Future work
• Developed Excel ingest
• Still working on PDF and Word ingest

# Phil 2.11.16

6:00 – 4:00 VTX

• Continuing Participatory journalism – the (r)evolution that wasn’t. Content and user behavior in Sweden 2007–2013
• Need to see if I can get this on Monday: Rethinking Journalism: trust and participation in a transformed news landscape. Got the kindle book.
• Need to add a menubar to the Gui app that has a ‘data’ and ‘queries’ tab. Data runs the data generation code. Queries has a list of questions that clears the output and then sends the results to the text area.
• Still need to move the db to a server. Just realized that it could be a MySql db on Dreamhost too. Having trouble with that. It might be the eclipse jar? Here’s the hibernate jar location in maven:
<groupId>org.hibernate.javax.persistence</groupId>
<artifactId>hibernate-jpa-2.0-api</artifactId>
<version>1.0.1.Final</version>
• Gave up on connecting to Dreamhost. I think it’s a permissions thing. Asked Heath to look into creating a stable DB somewhere. He needs to talk to Damien.
• Webhose.io – direct access to live & structured data from millions of sources.
• Google news search that produces Json for the last 24 hours:
?q=malpractice&safe=off&hl=en&gl=us&authuser=0&tbm=nws&source=lnt&tbs=qdr:d
• Played around with a bunch of queries, but in the end, I figured that it was better to write the whole works out in a .csv file and do pivot tables in Excel.
• Adding the ability to read a config file to set the search engines, lables, etc for generation.

Data Architecture Meeting 2.11.15

Testing what we have

• Relevance score
• Pertinence score
• Charts for management

Vinny

• Terminology
• gov
• Bias towards trustworthy unstructured sources.
• What about getting structured data.

Aaron

• Isolate V1 capability
• Metrics!
• We need the structured data!!

Matt

• Dsds

Scott