Category Archives: Users

Phil 8.18.17

7:00 – 8:00 Research

  • Got indexFromLocation() working. It took some fooling around with Excel. Here’s the method:
    public int[] indexFromLocation(double[] loc){
        int[] index = new int[loc.length];
        for(int i = 0; i < loc.length; ++i){
            double findex = loc[i]/mappingStep;
            double roundDown = Math.floor(findex);
            double roundUp = Math.ceil(findex);
            double lowdiff = findex - roundDown;
            double highdiff = roundUp - findex;
            if(lowdiff < highdiff){
                index[i] = (int)roundDown;
                index[i] = (int)roundUp;
        return index;
  • And here are the much cleaner results:
    • [0.00, 0.00] = [0, 0]
      [0.00, 0.10] = [0, 0]
      [0.00, 0.20] = [0, 1]
      [0.00, 0.30] = [0, 1]
      [0.00, 0.40] = [0, 2]
      [0.00, 0.50] = [0, 2]
      [0.00, 0.60] = [0, 2]
      [0.00, 0.70] = [0, 3]
      [0.00, 0.80] = [0, 3]
      [0.00, 0.90] = [0, 4]
      [0.00, 1.00] = [0, 4]

      [1.00, 0.00] = [4, 0]
      [1.00, 0.10] = [4, 0]
      [1.00, 0.20] = [4, 1]
      [1.00, 0.30] = [4, 1]
      [1.00, 0.40] = [4, 2]
      [1.00, 0.50] = [4, 2]
      [1.00, 0.60] = [4, 2]
      [1.00, 0.70] = [4, 3]
      [1.00, 0.80] = [4, 3]
      [1.00, 0.90] = [4, 4]
      [1.00, 1.00] = [4, 4]
  • Another thought that struck me as far as the (int) constraint is that I can have a number of ArrayLists that are embedded in a an object that has the first and last index in it. These would be linked together to provide unconstrained (MAX_VALUE or 2,147,483,647 lists) storage

8:30 – 4:30 BRI

  • I realized yesterday that the Ingest and Query microservices need to access the same GeoMesa Spring service. That keeps all the general store/query GeoMesa access code in one place, simplifies testing and allows for DI to provide the correct (hbase, accumulo, etc) implementation through a facade interface.
  • Got tangled up with getting classpaths right and importing the proper libraries
  • Got the maven files behaving, or at least not complaining on mvn clean and mvn compile!
  • Well that’s a new error: Error: Could not create the Java Virtual Machine. I get that running the new installation with the geomesa-quickstart-hbase
    • Ah, that’s what will happen when you paste your command-line arguments into the VM arguments space just above where it should go…
    • Wednesday’s goal will to verify that HBaseQuickStart is running correctly in its new home and start to turn it into a service.

Phil 8.17.17

BRI – one hour chasing down research hours from Jan – May

7:00 – 6:00 Research

  • Found this on negative flocking influences: The rise of negative partisanship and the nationalization of US elections in the 21st century.  Paper saved to Lit Review
    • One of the most important developments affecting electoral competition in the United States has been the increasingly partisan behavior of the American electorate. Yet more voters than ever claim to be independents. We argue that the explanation for these seemingly contradictory trends is the rise of negative partisanship. Using data from the American National Election Studies, we show that as partisan identities have become more closely aligned with social, cultural and ideological divisions in American society, party supporters including leaning independents have developed increasingly negative feelings about the opposing party and its candidates. This has led to dramatic increases in party loyalty and straight-ticket voting, a steep decline in the advantage of incumbency and growing consistency between the results of presidential elections and the results of House, Senate and even state legislative elections. The rise of negative partisanship has had profound consequences for electoral competition, democratic representation and governance.
  • Working on putting together an indexable high-dimension matrix that can contain objects. Generally, I’d expect it to be doubles, but I can see Strings and Objects as well.
  • Starting off by seeing what’s in the newest Apache Commons Math (v 3.6.1)
  • Found SimpleTensor, which uses the Efficient Java Matrix Library (EJML) and creates a 3D block of rows, columns and slices. THought it was what I wanted, but nope
  • Looks like there isn’t a class that would do what I need to do, or that I can even modify. I’m thinking that the best option is to use org.apache.commons.math3.linear.AbstractRealMatrix as a template.
  • Nope, coudn’t figure out how to do things as nested lists. So I’m doing it C-Style, where you really only have one array that you index into. Here’s a 4x4x4x4 Tensor filled with zeroes:
    Total elements = 256
    0.0:[0, 0, 0, 0], 0.0:[1, 0, 0, 0], 0.0:[2, 0, 0, 0], 0.0:[3, 0, 0, 0],
    0.0:[0, 1, 0, 0], 0.0:[1, 1, 0, 0], 0.0:[2, 1, 0, 0], 0.0:[3, 1, 0, 0],
    0.0:[0, 2, 0, 0], 0.0:[1, 2, 0, 0], 0.0:[2, 2, 0, 0], 0.0:[3, 2, 0, 0],
    0.0:[0, 3, 0, 0], 0.0:[1, 3, 0, 0], 0.0:[2, 3, 0, 0], 0.0:[3, 3, 0, 0],
    0.0:[0, 0, 1, 0], 0.0:[1, 0, 1, 0], 0.0:[2, 0, 1, 0], 0.0:[3, 0, 1, 0],
    0.0:[0, 2, 3, 3], 0.0:[1, 2, 3, 3], 0.0:[2, 2, 3, 3], 0.0:[3, 2, 3, 3],
    0.0:[0, 3, 3, 3], 0.0:[1, 3, 3, 3], 0.0:[2, 3, 3, 3], 0.0:[3, 3, 3, 3]
  • The only issue that I currently have is that ArrayLists are indexed by int, so the total size is 32k elements. That should be good enough for now, but it will need to be fixed.
  • set() and get() work nicely:
    lt.set(new int[]{0, 1, 0, 0}, 9.9);
    lt.set(new int[]{3, 3, 3, 3}, 3.3);
    System.out.println("[0, 1, 0, 0] = " + lt.get(new int[]{0, 1, 0, 0}));
    System.out.println("[3, 3, 3, 3] = " + lt.get(new int[]{3, 3, 3, 3}));
    [0, 1, 0, 0] = 9.9
    [3, 3, 3, 3] = 3.3
  • Started the indexFromLocation method, but this is too sloppy:
    index[i] = (int)Math.floor(Math.round(loc[i]/mappingStep));

Phil 8.16.17

7:00 – 8:00 Research

  • Added takeaway thoughts to my C&C writeup.
  • Working out how to add capability to the sim for P&RCH paper. My thoughts from vacation:
    • The agents contribution is the heading and speed
    • The UI is what the agent’s can ‘see’
    • The IR is what is available to be seen
    • An additional part might be to add the ability to store data in the space. Then the behavior of the IR (e.g. empty areas) would b more apparent, as would the effects of UI (only certain data is visible, or maybe only nearby data is visible) Data could be a vector field in Hilbert space, and visualized as color.
  • Updated IntelliJ
  • Working out how to to have a voxel space for the agents to move through that can also be drawn. It’s any number of dimensions, but it has to project to 2D. In the case of the agents, I just choose the first two axis. Each agent has an array of statements that are assembled into a belief vector. The space can be an array of beliefs. Are these just constructed so that they fill a space according to a set of rules? Then the xDimensionName and yDimensionName axis would go from (0, 1), which would scale to stage size? IR would still be a matter of comparing the space to the agent’s vector. Hmm.
  • This looks really good from an information horizon perspective: The Role of the Information Environment in Partisan Voting
    • Voters are often highly dependent on partisanship to structure their preferences toward political candidates and policy proposals. What conditions enable partisan cues to “dominate” public opinion? Here I theorize that variation in voters’ reliance on partisanship results, in part, from the opportunities their environment provides to learn about politics. A conjoint experiment and an observational study of voting in congressional elections both support the expectation that more detailed information environments reduce the role of partisanship in candidate choice

9:00 – 5:00 BRI

  • Good lord, the BoA corporate card comes with SIX seperate documents to read.
  • Onward to Chapter Three and Spring database interaction
  • Well that’s pretty clean. I do like the JdbcTemplate behaviors. Not sure I like the way you specify the values passed to the query, but I can’t think of anything better if you have more than one argument:
    public class EmployeeDaoImpl implements EmployeeDao {
        private DataSource dataSource;
        private JdbcTemplate jdbcTemplate;
        private RowMapper<Employee> employeeRowMapper = new RowMapper<Employee>() {
            public Employee mapRow(ResultSet rs, int i) throws SQLException {
                Employee employee = new EmployeeImpl();
                employee.setEmployeeName(rs.getString("FirstName") + " " + rs.getString("LastName"));
                return employee;
        public Employee getEmployeeById(int id) {
            Employee employee = null;
            employee = jdbcTemplate.queryForObject(
                    "select * from Employee where id = ?",
                    new Object[]{id},
            return employee;
        public List<Employee> getAllEmployees() {
            List<Employee> eList = jdbcTemplate.query(
                    "select * from Employee",
            return eList;
  • Here’s the xml to wire the thing up:
    <context:component-scan base-package="org.springframework.chapter3.dao"/>
    <bean id="employeeDao" class="org.springframework.chapter3.dao.EmployeeDaoImpl"/>
    <bean id="dataSource"
        <property name="driverClassName" value="${jdbc.driverClassName}" />
        <property name="url" value="${jdbc.url}" />
        <property name="username" value="xxx"/>
        <property name="password" value="yyy"/>
    <bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
        <property name="dataSource" ref="dataSource" />
    <context:property-placeholder location="" />
  • And here’s the properties. Note that I had to disable SSL:

Phil 8.15.17

7:00 – 8:00 Research

8:30 – 6:00 BRI

  • Remembered my bike shoes, but it may rain all day
  • Wrote up a note to T about burning PhD hours for the Oct1 CHIIR paper deadline
  • Need to write up the dev plan for GeoMesa into the EIP pattern (include a strawman use case?)
  • Back to learning Spring
  • And here is a way to populate a nested bean:
    <bean id="countryParent" abstract="true" class="org.springframework.chapter2.pojos.EmployeeImpl"><!-- this just gets one abstract class implementation -->
        <property name="country" value="USA"/>
    <bean id="employeeServiceBean"
        <property name="employeeList">
                <bean parent="countryParent" class="org.springframework.chapter2.pojos.EmployeeImpl">
                    <property name="employeeAge" value="10"/>
                    <property name="employeeName" value="Ten Tin"/>
                    <property name="employeeId" value="1"/>
                <bean parent="countryParent" class="org.springframework.chapter2.pojos.EmployeeImpl">
                    <property name="employeeAge" value="11"/>
                    <property name="employeeName" value="Eleven Even"/>
                    <property name="employeeId" value="2"/>
                <bean parent="countryParent" class="org.springframework.chapter2.pojos.EmployeeImpl">
                    <property name="employeeAge" value="12"/>
                    <property name="employeeName" value="Twelve Elve"/>
                    <property name="employeeId" value="3"/>
  • That sends the instanced objects to setEmployeeList that has to take a list of objects. To get it typed, I did this:
    public void setEmployeeList(List<Object> employeeList) {
        for(Object o : employeeList){
            Employee e = (Employee)o;
  • A typed employee is added as follows:
    public void setEmployee(Employee employee) {
  • IntelliJ will extract an interface and rename your class. How cool!
  • Printed, signed and scanned my credit card agreement.

Phil 8.13.17

8:30 – 3:30  BRI

  • Got in at 10:30 last night and am here on time. Kind of surprised. I’ll need to leave a bit early so that I can get shopping and lawn triaged. It rained a LOT while I was gone.
  • getting caught up, updating IDE, etc. It looks like I’m still working on GeoMesaIngest.
  • Since I still need to come up to speed on Spring, and I don’t have tasking, I think I’ll start there
  • Downloaded book
  • installed lamp-server^ and php. Verified that everything is running
  • Got an end-to-end Spring example running with no mysterious incantations. Nice. Now I want to understand testing, but I have to go through a few chapters first…

Phil 7.27.17 – 8.13.17

Vacation (Strava links below)

  1. Day 1 – 37 Miles (To Forest Grove)
  2. Day 2 – 83 Miles, parts one and two (To Astoria)
  3. Day 3 – 69 Miles to Tillamook
  4. Day 4 – 81 Miles to Newport
  5. Day 5 – 58 Miles to Florence
  6. Day 6 – 79 Miles to Bandon
  7. Day 7 – 80 Miles to Roseburg, parts one and two
  8. Day 8 – 102 Miles to Eugene
  9. Day 10 – 83 Miles to Willamette pass. Now that was a big hill
  10. Day 11 – 79 Miles to Bend, parts one and two
  11. Day 12 – 80 Miles from Bend to Detroit Lake
  12. Day 13 – 93 Miles to Troutdale
  13. Day 14 – 23 Miles to Portland

A grand total of about 947 Miles, or an average of 72 miles a day. Fun!

Also other interesting things:

Thought from rest day one:

  • It mak not be a map in the classical sense of a representation of fixed points. The representation may more resemble the detailed pictures that have been coming in from Cassinii recently. There is no ground, but there are large scale recognisable features (the Red Spot, bands, hexagonal poles, etc). The relationships between the features can change, but the patterns could be stochastic in the sense that there is a sustaining pattern, like a candle flame. From C&C:
    • Every kind of intellectual and affective means come into play as soon as the slightest hint of difference arises, so that arguments are invented, opposing views interpreted, to find the grounds for disagreement, but, above all to find a way out of them. [p 173]  pia07782_hires
  • For the simulation
    • The agents contribution is the heading and speed
    • The UI is the horizon
    • The IR is the stage
    • An additional part might be to add the ability to store data in the space. Then the behavior of the IR (e.g. empty areas) would b more apparent, as would the effects of UI (only certain data is visible, or maybe only nearby data is visible) Data could be a vector field in Hilbert space, and visualized as color.

Rest Day 2

  • C&C
    • By causing the majority to converge towards a social representation, the multiplicity of decisions that lead to a consensus do more than draw viewpoints closer: they initiate or reinforce social ties. Beyond the specific dilemmas that each decision resolves, they correspond to this general aim. They produce a mass effect in the network of groups that choose and discuss, creating and re-creating the bonds in our society by a common action, just as at one time public opinion originated in the market-places and cafes, and from drawing-room conversations. It is the surplus value that we extract from this task of collective decision-making, the diversity and multiplicity of which have reached so high a level that it has become a profession -we speak of decision-makers – and a significant factor in our social and moral world. [p 174]
      • Is community building inherently an act of dimension reduction? We find commonalities and focus on those, while excluding other areas. It’s not necessarily about agree/disagree, but more like areas that we can create norms of behavior in. It’s easier to create norms in homogeneous, closed cultures, and harder in heterogeneous cultures. But though homogeneous cultures may be faster to respond, they are also less resilient. I’m thinking in particular of the Aspen, or any plant that reproduces asexually. The there is an initial advantage, but if conditions change, the plant goes extinct quickly.
    • The socio-cognitive conflict inherent in any decision taken in common combines together two permanent tendencies. The one aims at maintaining existing uniformity and agreement, the other at changing them by imparting an original form to things and ideas. The choices that are made usually express a balance of forces between the two because, without any element of novelty, they are mere stereotypes or ritual, and, without a dose of conformity, they become fancies and fluctuations that lead to disorder. It is of real theoretical interest to recognize at work in the socio-cognitive conflict a dual process of social thought. It can only be dual, in.view of the opposing functions that it fulfils and of the simultaneous use it makes of divergent and convergent thinking, the one the badge of innovation and the other of uniformity [p 174]
      • This is a very good example of the explore/exploit condition as it relates to social cognition. The same for sea turtles and people
    • To resolve the conflict by eliminating convergent thinking would be to abandon discussion and any choice made in common. To resolve the conflict by censoring divergent thinking would condemn participants to routine, to stereotyping, to what is termed ‘groupthink’. On the other hand, to negotiate this conflict, which is both social and cognitive, is an arduous task. [p 175]
    • The effects of a normative intervention on group decision-making performance
      • A space ship having crashed on the moon, a team of astronauts has to cover a distance of some 300 kilometres in order to reach the spot where they have a rendezvous with another team. Before embarking on this perilous undertaking, the members of the team have to decide which of the fifteen objects necessary for survival – oxygen reserves, concentrated food, signalling equipment, heating requisites, etc. – they will take with them. Those participating in the study were asked to draw up a list of priorities for these objects, first separately as individuals, and then in groups, by arriving at a consensus. Half of the groups received no special instructions for this common task of decision-taking. The other half were instructed to confront other members of the group and resolve the differences between them. They were informed that they had to set out their arguments lucidly and were not to change their opinion with the sole aim of avoiding conflict, nor to seek agreement using procedures such as a majority vote, the establishment of a mean position, bargaining, tossing a coin, or in other ways. Moreover, the instructions emphasized the need to look upon differences of opinion as both natural and useful, so that any precipitous agreement would be suspect, so long as the reasons for it had not been gone into thoroughly. The main thing was to resist group pressures that tempted one to yield to others without sound reasons, just in order to attain a consensus, which would lack any guarantee of success. [p 176]
        • Good game for study design?
      • Hall and Watson were persuaded that by weakening such pressures they would encourage divergent thinking to be displayed. This would lead groups first to produce solutions of a superior quality, and then to make better use of the resources of each member, so that overall performance would exceed even that of the group’s cleverest members, and finally to discover novel solutions. Most of these hypotheses were verified.
    • The other means of encouraging divergent thinking is by the presence of a minority participating actively in the group’s discussions. This is a comparatively natural means for use in decisions leading to a consensus, since it arises solely from the obligation to respect one of the essential conditions. These assume in fact a state of equality between the members of the group. This means the majority recognizes the right of the minority to express itself, and will set very great store by its opinions; otherwise the agreement arrived at would be worthless. [p 177]
    • (Moscovici and Zavalloni, 1969) looked for indications of this cognitive transformation of individuals in a group. It is no exaggeration to state that the decisions, in the broadest sense, prepare the way for it and make it their prime target. The students who participated, it will be recalled, had to adopt a common attitude towards de Gaulle and the Americans. In order to arrive at this, particularly if the attitude were unfavourable, they had to acknowledge a common set of values and a common code. This most possibly assumed , among the categories available to each individual, the use solely of those that corresponded to that code. These were the ones that were retained, that were used frequently, and towards which individuals would converge. In order to verify this we calculated an entropy index (H) of the distribution of categories, borrowed from information theory. [p180]
      • This looks like dimension reduction to me. It also implies a way of testing for it. The description for how they calculated entropy isn’t that clear from the book, but I’m guessing that it has something to do in the variability of the terms used. Combined with an awareness of sentiment, this might be a reasonable way to determine relevant axis of discussion

Phil 7.26.17

7:00 – 8:00 Research

  • A quick thought about the overlapping of spatial and conceptual ‘space’ concepts. Dug up the studies that showed how physical warmth can cause social ‘warmth’ and some of the underlying neural structures
    • Experiencing Physical Warmth Promotes Interpersonal Warmth
      • “Warmth” is the most powerful personality trait in social judgment, and attachment theorists have stressed the importance of warm physical contact with caregivers during infancy for healthy relationships in adulthood. Intriguingly, recent research in humans points to the involvement of the insula in the processing of both physical temperature and interpersonal warmth (trust) information. Accordingly, we hypothesized that experiences of physical warmth (or coldness) would increase feelings of interpersonal warmth (or coldness), without the person’s awareness of this influence. In study 1, participants who briefly held a cup of hot (versus iced) coffee judged a target person as having a “warmer” personality (generous, caring); in study 2, participants holding a hot (versus cold) therapeutic pad were more likely to choose a gift for a friend instead of for themselves.
    • Shared neural mechanisms underlying social warmth and physical warmth.
      • Many of people’s closest bonds grow out of socially warm exchanges and the warm feelings associated with being socially connected. Indeed, the neurobiological mechanisms underlying thermoregulation may be shared by those that regulate social warmth, the experience of feeling connected to other people. To test this possibility, we placed participants in a functional MRI scanner and asked them to (a) read socially warm and neutral messages from friends and family and (b) hold warm and neutral-temperature objects (a warm pack and a ball, respectively). Findings showed an overlap between physical and social warmth: Participants felt warmer after reading the positive (compared with neutral) messages and more connected after holding the warm pack (compared with the ball). In addition, neural activity during social warmth overlapped with neural activity during physical warmth in the ventral striatum and middle insula, but neural activity did not overlap during another pleasant task (soft touch). Together, these results suggest that a common neural mechanism underlies physical and social warmth.
  • Looked more closely at the NSF grants. I think the mapping fits the following:
  • Information Integration and Informatics (III)
    • Technological advances have resulted in accelerating increases in size, diversity, and complexity of data in virtually all aspects of human endeavor. Our ability to gather data of all types greatly outstrips our cognitive capacity to use it,while scientific, technical, and societal advances are increasingly dependent on new insights, theories, and tools to exploit data effectively for timely delivery of relevant and accurate information and for knowledge discovery. The Information Integration and Informatics (III) program supports research to realize the full transformative potential of data, information, and knowledge in this increasingly digital and interconnected world.
    • III-funded projects may address data of unprecedented scale, complexity, and rate of acquisition, as well as issues of heterogeneity and complexity with innovative approaches and deep insights. Projects may support the diverse functionalities and processing needs for data, information, and knowledge from disparate and uncoordinated sources,or cope with the changing landscape of computing platforms at scales ranging from small mobile devices to potentially global-scale cloud and networked computing resources. Successful proposals should demonstrate effectiveness in dimensions such as scalability, interactivity, or scientific, technological, or societal impact.
    • III-funded projects should address contemporary applications of societal importance through advances in information integration and informatics. Projects may deal with one or more facets of the full knowledge life-cycle, including creation, acquisition, selection, storage, display, and preservation, use, and re-use of data, information, and knowledge for decision-making and action. Ultimately, the deep scientific insights and advanced technologies resulting from III-funded projects will transform the functions and uses of data, information, and knowledge in society.
    • More information on topics of interest to the III program is available at:
  • More C&C
    • …certain aspects of a situation that bear no relationship to one another, such as the dimensions of a room and the decisions taken in it, that is, aspects deemed to be ancillary and which apparently should not be significant, may in reality be determining factors. In other words, what one considers to be a factor of the world external to social relationships has indeed an action within them. [p 130]
      • Design matters in all kinds of ways, and can affect the outcome and quality of the result. Consider polling – focus group, personal home interview, phone, online. What kind of bias do these modalities introduce? There was more detail on p 131 as well
    • In the case where different opinions can be freely expressed, discussion allows each individual better to defend his own viewpoint and to become more involved in the activity of the group. Thus we expected that the consensus would stress the dominant tendency among its members – in short, be polarized. On the other hand, when the participants placed a distance between one another and paid attention to the manner in which they arrived at a decision, their attention was concentrated upon positional differences. The discussion of the latter could not involve them greatly. Moreover, they had no means of justifying or forcibly making their position prevail, and thus were thrown back on resolving their differences by a compromise. This is indeed what we observed. Consequently attention to rules of procedure, which direct discussion towards information regarding the responses, leads towards a decision of ‘the just mean’. Discussion directed towards content produces the opposite effect. [ p 136]
      • This makes me a little nervous about using Pro Publica congressional data, since the law is very process oriented. Still, the body does seem to be polarizing, and potentially in different ways. Republicans are moving to the right and also disrupting process. Democrats appear to be moving less and have been more focused on ‘standard procedure’. Something to consider is if this is a function of being the opposition party of the party in power. The incentive when in power is to do things, which may require more compromise, as opposed to being the opposition party, which is almost by definition, uncompromising.
    • Imposing time limits on discussions also leads to compromise, while having open-ended discussions tend to polarize. The time constraint acts as a limiter of possible options? [p 136-137]
    • …groups that had to deal with information according to a set method, following directions, took compromise decisions. By contrast, with groups who could deal with the information without being subject to any constraints, the decisions were more extreme than the individual decisions. In principle we have here a confirmation of what we had always supposed to be the case: a normalized form of participation, which involves individuals less, leads to a moderate decision[p 138]

8:30 – 4:30 BRI

  • Finished stories
  • Built and uploaded MDS, NLP, Geocoder and GeoIngest
  • Had a variety of problems getting GeoIngest tests to behave. I need to come up to speed on Spring, I think. At least if this is going to be a regular thing.

Phil 7.25.17

Finally a cool, lower humidity day! The windows are open and the AC is off.

7:00 – 8:00 Research

  • The Ring Theory of Kvetching has a different, “kvetcher-centered” map
  • Inglehart–Welzel cultural map of the world  inglehart-welzel_2015
  • The spread of fake news by social bots
    • The massive spread of fake news has been identified as a major global risk and has been alleged to influence elections and threaten democracies. Communication, cognitive, social, and computer scientists are engaged in efforts to study the complex causes for the viral diffusion of digital misinformation and to develop solutions, while search and social media platforms are beginning to deploy countermeasures. However, to date, these efforts have been mainly informed by anecdotal evidence rather than systematic data. Here we analyze 14 million messages spreading 400 thousand claims on Twitter during and following the 2016 U.S. presidential campaign and election. We find evidence that social bots play a key role in the spread of fake news. Accounts that actively spread misinformation are significantly more likely to be bots. Automated accounts are particularly active in the early spreading phases of viral claims, and tend to target influential users. Humans are vulnerable to this manipulation, retweeting bots who post false news. Successful sources of false and biased claims are heavily supported by social bots. These results suggests that curbing social bots may be an effective strategy for mitigating the spread of online misinformation.
  • The Age of the Crowd: A Historical Treatise on Mass Psychology Looks like a mass audience book by Moscovici. The table of contents is very compelling:
    • Part I. The Study of the Masses:
      1. The individual and the masses;
      2. The revolt of the masses;
      3. What do we do when faced with the masses?;
      4. Eastern and western varieties of despotism;
    • Part II. Le Bon and the Fear of Cowards:
      1. Who was Gustave Le Bon?;
      2. The Machiavelli of mass societies;
      3. Four reasons for saying nothing;
      4. The discovery of the masses;
      5. Mass hypnosis;
      6. The mental life of crowds
    • Part III. The Crowd, Women and Madness:
      1. Collective matter: the impulsive and conservative crowd;
      2. Collective form: the dogmatic and utopian crowd;
      3. The leaders of the crowd;
      4. Charisma;
      5. The strategies of propaganda and mass suggestion;
      6. Conclusion;
    • Part IV. The Leader Principle:
      1. The paradox of mass psychology;
      2. Natural crowds and artificial crowds;
      3. The leader principle;
    • Part V. Opinion and the Crowd:
      1. Communication is the valium of the people;
      2. Opinion, the public and the crowd;
      3. The law of the polarisation of prestige;
      4. The Republic in France: from a democracy of the masses to a democracy of publics
    • Part VI. The Best Disciple of Le Bon and Tarde: Sigmund Freud:
      1. The black books of Dr Freud;
      2. From classical to revolutionary mass psychology;
      3. The three questions of mass psychology;
      4. Crowds and the libido;
      5. The origin of affective attachments in society;
      6. Eros and mimesis;
      7. The end of hypnosis;
    • Part VII. The Psychology of the Charismatic Leader:
      1. Prestige and charisma;
      2. The postulate of mass psychology;
      3. The primal secret;
    • Part VIII. Hypotheses About Great Men:
      1. ‘The man Moses’;
      2. The family romances of great men;
      3. Creating a people;
      4. Mosaic and totemic leaders;
    • Part IX. Secular Religions:
      1. The secret of a religion;
      2. The prohibition of thought;
      3. The cult of the father;
    • Conclusion: the planetary age of the crowd;
  • More C&C
    • Inevitably we will end up by acknowledging the place of values in a world of information. Indeed the problem is to know which items of information to use ‘and which to discard, in order to arrive at a particular agreement. [p 122]
      • To restate, how to perform manifold reduction from a high dimensional, incomprehensible problem to something that is a more manageable size. This can include combining dimensions, deleting dimensions and synthesizing new dimensions that may or may not have a direct connection to the problem but seem relevant (Blindly following a leader, for example).
    • We may say that values are their vertical and horizontal coordinates: Even if they are not rational, they make reason possible, reason which for us has – and we must not forget it- the meaning of a value or norm, the highest norm to which our society pays homage. Hence the requirement that is emphasized by two specialists in decision-making; it is ‘a theory of rational consensus and commitment when, in fact there is a dissensus (Lehrer and Wagner, 1982: 4). To fulfil this need, such a theory, as we have demonstrated, must establish a relationship between these ideas. [p 122]
      • Self-organizing value maps, with axis based on dissensus measures. I think this is what I’m aiming for. And I think that dissensus could be measured by looking for the broadest range of sentiment applied to words, phrases or entities. These axis may be organizable using betweenness centrality measures that keep like axis together.

8:30 – 5:00 BRI

  • So the problem I’ve been having with the GEM was that JsonNode.asText() returned an empty String. JsonNode.toSting() does what asText is supposed to do as I understand it… Anyway, I am now able to generate a json message that can be used to test the GeoMesaIngestService.
  • Finished out the Web ingest test
  • Started the service ingest test
  • Need to enter stories and flesh out queries
  • Need to push NLP, Geocoder and Ingest before heading out today
  • Timesheet!

Phil 7.24.17

I seem to be making $800 mistakes these days. Got the flight information wrong on the flights to HCIC, and this past Saturday I crashed my bike on wet roads, which destroyed the camera I was carrying and my front brake. Here’s hoping that these things don’t happen in threes…

7:00 – 8:00 Research

  • Codev2 by Lawrence Lessig:Lessig’s “Code and Other Laws of Cyberspace” was published in 1999. The book quickly began to define a certain vocabulary for thinking about the regulation of cyberspace. More than any other social space, cyberspace would be controlled or not depending upon the architecture, or “code,” of that space. And that meant regulators, and those seeking to protect cyberspace from at least some forms of regulation, needed to focus not just upon the work of legislators, but also the work of technologists.Code v2 updates the original work. It is not, as Lessig writes in the preface, a “new work.” Written in part collectively, through a Wiki hosted by JotSpot, the aim of the update was to recast the argument in the current context, and to clarify the argument where necessary.
  • More C&C
  • This is tantamount to stating that the social universe resembles the organic universe of Aristotle, in which there can be distinguished a centre and a periphery, a top and a bottom, and a high and a low, rather than the mechanical universe of Newton, homogeneous and lacking any one favoured direction. Without wasting words, we may state that in decision making there is no tabula rasa, any more than there are many decisions that are disputed once they have been taken. [p 113]
  • …consensus polarized in the direction of emerging norms [p 114]
    • Are norms poles that have a position and velocity in belief space? Or are they a manifestation of the group behavior, sort of the position of the average future center (position and variance?) of the group. In other words, do norms have an attraction or are they an implicitly agreed on, emergent, set of beliefs that exist at a certain time? And along these lines, are there patterns that persist over different time spans? I would bet that a circling flock has a centroid of (persistent beliefs), while a stampede doesn’t. This ties back to Arednt’s description of totalitarianism and terror where change is the only constant.
  • On the whole here we are looking at, broadly sketched out, the picture of what must occur when a problem evokes a large-scale movement of opinion. People participate in the debate frequently and with intensity. The series of decisions leading to consensus polarizes towards the norm that is emerging and, by this very fact, emphasizes it. Thus these decisions cause the norm to crystallize and facilitate its being embraced fully by each individual, who feels himself a little its creator. Therefore no coercion or forced consensus should enter into it. [p 117]
    • Support for the idea that the norms are not a pole, but a projection of the future?
  • Attitude polarization, familiarization and group process
  • What happened where a feminist or anti-feminist confederate was present? Of course both were confronting an attitude that had crystallized, had become solidly fixed, and was almost a cultural cliche. The feminist confederate was no longer proposing anything new, but was defending what had become a norm and his or her influence was reinforcing conformity, whereas the anti-feminist confederate seemed to be utterly conservative, a reactionary deviant. Upon examining the data, it was found that their influence had little room in which to be exerted, and that it was weak. Thus the feminist confederates succeeded in polarizing somewhat during the discussion, but to a significant degree (F = 6.56;p < .01), the consensus. On the other hand, the confederate defending an anti-feminist position brought about no reaction, as if this was already ruled out (see Table 5.4). This is why one can no longer observe the former bi-polarization, when the discussions on the status of women set pro-feminists against their adversaries. At any rate the heated atmosphere had cooled down. The researchers noted that there were fewer arguments, and that the discussions were flat and unenthusiastic. An air of nostalgia hung over the groups, who seemed to be asking themselves, ‘But where are the debates of yesteryear?’, just as the poet Villon had once asked, ‘Where are the snows of yesteryear?’ They knew they were waging a fight that had been won by others quite a time ago. [p 120]
    • This would be the old study, where norms are emerging (beliefs in collision): UnstableFlockFormation This would be stable, evolving norms: StableFlock And this would be something like a fixed ideology FixedIdeologyStampede
    • Here’s some charts of ideology in Congress. Picture links to article. The variance is interesting – it implies in the chart on the left that democrats are becoming less diverse, while republicans seem to be breaking into sub-populations(?). polarization

9:00 – 5:00 BRI

  • Need to write up stories
  • Need to discuss refactoring GeocoderService
  • Stories
    • Mock tests for the interface 3 points (fix unit tests)
    • Modify GeoIngestService for document-centric storage of GEM Facts – 5 points
    • Refactor GeoMesaIngestService to GeoMesaDatabaseService – 5 points
    • Business logic for publicly accessible methods
      • Store (produce stunt GEM Json) – 5 points
      • Queries (8 points)
        • By geo coordinate
        • By document and geo coordinate
        • By location name and geo coordinate
      • Deploy – 3 points
  • Wrote teh above up as an email and sent off to Matt
  • Working on getting the json assembled but the serializer is choking on a null elevation. There is a way to get elevation data from Google, but it requires a separate call. Stuffing in zeros for the time being. Oops:
  • private double latitude;
    private double longitude;
    private Double elevation;
  • No, that’s a feature, but the serializer chokes. Not sure what to do and it looks like everyone has left?
  • {
     "name": "elevation",
     "type": ["null", "double"], <- serializer does not like this
     "default": null,
     "doc": "The elevation in meters. May be negative, 0, positive, or null, if unknown."
  • Conversation with Gerard:
    • pfeldman 10:38 AM
      Ok – here’s the issues:
    • Currently Ingest stores people. It can’t NLP provides location information with associated document ID in the GEM message
    • That means that GeoMesaIngest needs to store document and entity information associated with lat/long
    • This can be in GEM format, but that’s a more in-depth discussion
    • The other question is retreival
    • Within the current design, the Threat Assessment Service (TAS) will make a request of the geographic database about information ssociated with a person of interest.
    • The TAS (and the MDS) is where all the information is stored, so the TAS knows how to connect people and documents
    • So for a Geofence request about a person might be “give me all the location names that you have associated with this document” or “give me all the documents that are associated with this location name” or give me everything within this square
    • Is this the sort of query that the IngestService should respond to? Or is this handled somewhere else (GeoServer?)
    • gbriones 10:45 AM

      ingestservice should only worry about ingesting data, nothing else
    • the query should be handled elsewhere
    • pfeldman 10:46 AM

      Then stories need to be written to describe this interaction
    • So then my stories are as follows:
    • 1) Fix GeoCoder and Ingest so that they follow the EIP pattern of interface/implementation that allows for proper unit testing
    • 2) Fix GeoIngest so that it no longer takes in a person Json, but some variant of document/location name/lat-long
    • I think the refactoring is about 3 points, but the second is a good deal of work. I’ll probably need to ask some questions about the best way to store the data to get an idea of how much work it will actually be
    • And then you need the heads up that there is no person information in the GeoDB, just what I listed above.
    • Thoughts?
    • gbriones 10:51 AM

      that makes sense, i’m hoping you’ll have better leverage on getting this working
    • pfeldman 10:53 AM

    • gbriones 10:54 AM

      i am not knowledgeable enough of the architecture to be able to do much of anything for hooking into it
    • james didn’t have the knowledge either, which is apparent lol
    • i find it strange that the code passed review though
    • sean took a look at it a while ago, but i never heard back from him
    • pfeldman 10:55 AM

      Sean doesn’t know the architecture. Matt is the guy for that.
    • gbriones 10:55 AM

      ah, i see
    • pfeldman 10:56 AM

      Yeah, sorry, lots of hidden knowledge
    • If you take a look through confluence, the people who are the authors of the documents tend to be the owners of that knowledge, so that’s a way of figuring out who is responsible/knows what
    • So I would propose changing GeoMesaIngest to a more general service that handles ingest and query response. They are related, and Ingest is a very lightweight thing, even for a microservice
    • gbriones 11:00 AM

      ok, what do you mean by query response? just for clarification
    • pfeldman 11:00 AM

      The TAS will need to make queries of the geographic database
    • By geo coordinate
      By document and geo coordinate
      By location name and geo coordinate
    • something like a LL/UR area
    • gbriones 11:01 AM

    • pfeldman 11:03 AM

      I think it always returns a list of geolocation/location name/document ID tuples(?), though in reality it may put everything into GEM format
    • But the query has geocoordinate (LL/UR) plus one optional item
    • document or location name
    • maybe wild cards?
    • San F* is the example I was thinking, but that implies a more interactive interface. Is that more of a GeoServer thing?
    • gbriones 11:07 AM

      yeah, i believe it would be
    • pfeldman 11:08 AM

      So how is it thought that GeoServer will work? What are teh scenarios that have been discussed? Is anything written down?
    • gbriones 11:09 AM

      no, i think there will be talks for it, but nothing is currently planned or written down
    • we currently have geoserver up in dev as a way to look at the data on a super simple map
    • pfeldman 11:09 AM

      Can you send a screenshot? Curious
    • gbriones 11:10 AM

      no real ideas of how to integrate the visual aspect into the product right now though
    • let me see if it’s up, i can send you a link if it is
    • pfeldman 11:10 AM

      I am pretty sure that the customer wants to see maps, but there is no requirement *at all*
    • Soooo, back to stories
    • gbriones 11:11 AM

      yes, maps are part of the end goal, but the way of putting the map there is not fleshed out
    • we can use geoserver as a backend (with geomesa) to power the mapping
    • pfeldman 11:11 AM

      But how does it get data that is *only* in the MDS?
    • gbriones 11:12 AM

      the data will have to be ingested into geomesa
    • there’s a geomesa plugin for geoserver that will allow geoserver to look into that data and display it in layers
    • pfeldman 11:12 AM

      Nope. Not going to happen. Duplicate data will break
    • gbriones 11:12 AM

      that’s an issue that we brought up a long time ago
    • pfeldman 11:12 AM

      To who? What was the answer?
    • gbriones 11:13 AM

      here’s the integration document that matt drew up a couple months ago
    • we had a TEM with him about it around the time it was posted and discussed the issues with the approach outlined
    • i agree with you, duplicate data isn’t what we want here
    • but with the current architecture, i’m not sure if there’s a cleaner way of doing it
    • pfeldman 11:14 AM

      Yeah, I’m working from Phase 2 of this doc
    • Note the connection of GeoMesa to MDS in Phase 3
    • gbriones 11:14 AM

      yeah, we brought up the concerns then as well
    • pfeldman 11:16 AM

      So, knowing very little about GeoServer, could it handle a federated query from GeoMesa and MDS?
    • gbriones 11:17 AM

      geoserver should be able to handle that
    • pfeldman 11:18 AM

      Ok, then that’s not a Thing To Worry About Now
    • And back ones more to stories


      pfeldman 11:21 AM

      Wow! That’s INCREDIBLE! I’ve never seen such a collection of red dots
    • gbriones 11:22 AM

      tis a beautiful sight
    • pfeldman 11:22 AM

    • ok, back to stories and points
    • Mock tests for the interface 3 points
      Modify GeoIngestService for document-centric storage of GEM Facts – 5 points
      Refactor GeoMesaIngestService to GeoMesaDatabaseService – 5 points
      Business logic for publicly accessible methodsStore (produce stunt GEM Json) – 5 points
      Queries (8 points)
      By geo coordinate
      By document and geo coordinate
      By location name and geo coordinateDeploy – 3 points
    • thoughts?
    • gbriones 11:25 AM

      that sounds good
    • where are you working on this? machine wise?
    • pfeldman 11:26 AM

      All right, I’ll write those up and submit them on the backlog. I think I’m only going to be able to do the initial refactor to support the EIP pattern before I leave for Oregon. I’ll be back on the 13th, having forgotten *everything*

Phil 7.21.17

7:00 – 8:00 Research

  • Continuing C&C
  • to establish, on the one hand, the existence of a positive correlation between the average of individual positions and the shift towards the mean of positions around which a consensus is formed, and, on the other hand, the reduction in variance between the different positions after discussion in groups. [p 106]
    • echochambertest My model, All Exploit, stampede settings
  • polarization depended not upon the inclination of individuals to take risks or to remain prudent, but indeed on the form in which dilemmas are couched and debated, which causes the balance to swing one way or another. [p 106]
    • This is an example of the presentation affecting the outcome. If this were a gatekeeper issue, the information presented would have been different
  • Cvetkovitch and Baumgardner show that “group interaction will increase the participants’ involvement with the discussed topic and eventuate in individual opinion and group consensus that is more extreme than pre-discussion attitude. Additionally, the direction of polarization will be towards the naturally occurring majority opinion of the salient reference group and not toward the average attitude within the discussion groups. (1973: 161)” [p 107]
  • Main and Walker (1973) noted that in this respect the decisions were more liberal when the judges decided as a body (55 per cent as against 45 per cent). Knowing that only a minority of decisions taken by a judge presiding alone were of a liberal kind, the authors hypothesized that judges nevertheless had a liberal code of values. When they decided alone, the pressure of anti-liberal public opinion led them to a compromise. On the other hand, when they judged collegially their personal values came out and became more radical during discussion with their colleagues, who were of the same persuasion. [p 111]
    • A thought about how interacting with different groups with different norms might help reduce polarization. The flip side of this is that social media allows us to continually be present in a group online. We don’t visit in the same way as going to a meeting, social hour, gathering of friends, etc.

9:00 – 5:00 BRI

  • Aaron is still beating on the @AutoMiswired issue. It turns out that the GeoMesaIngest didn’t have the (interface) (method), which was causing Spring to point at the wrong (main) instance. Fixed!
  • Sprint review at 11:00. Looking for ways to present screenshots in rocket
  • Wound up not presenting anything. It looks like you have to fill out an agenda on Confluence
  • Sprint planning meeting. Whoops, didn’t make the tickets that describe the remaining work.
    • Create stories for deployment by DevOps (Heath) of GeoCoder and GeoMesaIngest
  • Something is wrong with the clicking on the document line – I’m getting an ‘IndexOutOfBoundsException’…