Phil 7.20.17

Dentist!

7:00 – 8:00 Research

  • Hybrid Forecasting Challenge from IAPA looks like something that the Research Browser might be good for. Lots of good proposer material here as well
  • Back to C&C
  • risky shift
  • they demonstrated this in a very simple way. On the one hand, they devised a questionnaire with choice dilemmas analogous to that used by the American social psychologists. Four dilemmas brought into play risk values; four others called for prudence values. But they changed the method of answering by using – as we did – a graduated seven-point scale. This read as follows: 1: strongly recommend x, the risky solution; up to 7: strongly recommend the prudent solution; with 4 as the neutral point at which both solutions appear of equal value.
  • The scale provides a measure of the tendency of the group, indicating unambiguously whether the average of the attitudes of its members at the beginning was located on the side of prudence or of risk. In order to verify the hypothesis, consensuses must be polarized in the direction of the initial values prevailing before the discussion, that is, they should not go beyond the neutral point, crossing the invisible Rubicon of risk towards prudence, or vice versa. The answer can therefore easily be given, since this neutral point expresses indecision or psychological indifference. It is clear that in most cases the groups have polarized, if one makes the comparison between the consensuses or final decisions and the initial decisions. Very rarely did they go beyond this neutral point in the opposite direction to that which their members had tended at the outset. If these members were bold, the shift took place in the direction of risk, and if they were prudent at the beginning, they became even more so. [p 101 -102]
  • Gouge and Fraser (1972) instead of choice dilemmas, proposed to the groups that they debate a great variety of problems ranging from drugs to sexuality, and including racism, suicide, etc. With one single exception, consensus accentuated the initial tendency of attitudes and judgements. Those propositions about which participants were already in agreement separately secured even greater agreement after discussion. Those that met with moderate agreement from individuals separately produced a more extreme agreement when they assembled in a group. [p 102]
    • There is a selection of the dimension(s) that the discussion will take place over.
    • There has to be a level of agreement along those dimensions
    • Closely aligned dimensions are therefore more likely to be in agreement, while further ones are less likely, since the ‘motion’ has less (implicit?) correlation.
    • Collapsing dimensions makes for easier discussion. If everything could be collapsed to one dimension, then it’s trivial (Arednt suggests that this is what totalitarianism is)
    • In the absence of external influence, with a sufficiently small number of dimensions, direction stays fixed, though I’m not sure about velocity. But this is echo chamber/stampede space
    • Diversity can be the presence of different headings, opposing velocity or additional dimensions (enlarging the information horizon)
  • depending on whether individuals are nearer or further away from the dominant pole of the scale, things proceed differently. When nearer to it, the extremists maintain their position, shifting less than do the moderates. This arises because the extremists can move only in a direction running counter to the norms, which is an eventuality ruled out, whereas the moderates can shift closer to this pole. As regards the other pole, it is the extremists who change more than do the moderates (p <-.001). They are comparatively more numerous than the latter (70 per cent as against 59 per cent) in linking up with the predominant norm in the population. Moreover, their greater distance from the norm has the result that after the discussion not only do they change in greater proportion, but also this change occurs to a significant degree. [p 104]
    • This implies that there is a physics model with respect to normative poles, though it may not fall off with distance.

10:30 – 5:00 BRI

  • Flailing at reworking GeoMesaIngest. Starting at the web test that fires the GeoMesaController with  mocked GeoMesaIngest. Except that’s throwing errors. Aaron is looking into it, says it seems familiar. Essentially, Spring needs to scan the framework to find the annotations, but the the test code is not being scanned properly, so code from the main is being injected into the test, which is breaking things.
  • Helped Aaron out a bit looking for projects that would serve as good frameworks to hang our research and demo work on

Phil 7.19.17

7:00 – 8:00 Research

  • Never re-OCR a PDF text in Acrobat. It screws up all the highlighting, because Adobe.
  • Based on what I’m reading in C&C, I think the next experiment might be to look at how to facilitate and map discussion groups. Roughly the same research browser concept – provide support for search in a variety of affordances, but also have a chat room where the discussions take place. Supporting points from user searches could be shared with other discussants. A copy of the annotated web page (highlights, etc) would appear as a new tab in the other browsers as a ‘shared from’. In this way, the important axis of the discussion could emerge and be factored into the map.
  • Ok, back to the reading. These were an important couple of pages:
  • It justifies our emphasizing how obsolete are the distinctions between factual and value judgments, the one exercising an informative influence, and the other a normative influence. Yet others continue to adhere to these distinctions. [p 97]
  • Each time the assessment is arrived at by ordering the terms on a scale, one of whose poles has in our eyes greater importance than the other. It is in relation to this pole that we place persons, things or ideas. [p 98]
    • This is what I was discussing above. It should be possible to determine what these poles are, and create maps using them. These maps will have utility, because they reflect what is important in the discussion.
    • A second issue is to be able to show the pattern where what is important changes. A timeline of pole eigenvectors might be helpful.
  • In a social environment, as soon as this hierarchy becomes explicit, individuals and groups clearly move towards the dominant pole. They seek to become more than they were, and this more so than others: more loyal, more courageous, more tolerant, more patriotic, more modern, and so on. This occurs particularly in novel circumstances, where experience does not relativize values or the image individuals wish to have of themselves. [p 98]
    • So, what does explicit mean here?
  • This is most certainly very apparent nowadays, where the value assigned to novelty, to being avant-garde, to the fact of being different, is very high. [p 98]
    • I think this is the motion part that leads to flocking and other dynamic behavior (e.g. fashion trends). Novelty is always attractive, because as animals that had to hunt and gather, we understand at a genetic level that stasis means that we starve.
  • the normative hypothesis of the theory: the tendency defined by the dominant values and attitudes is accentuated during the discussions, meetings, etc., and determines the directional shift of the decisions that lead to group consensus. [p 98]
  • To sum up: it may seem strange that groups spontaneously swing away from the just mean and the conformity they should adhere to. But they do not do so regardless of the direction, which is towards the norm to which they all adhere. This is why one can scarcely ask whether a consensus is going in the right or the wrong direction, without asking on what basis it has been established and by whom. Stated in statistical terms, this hypothesis predicts that the mean of the choices on which the group members reach agreement is closer to the dominant pole of its scale of values than the mean of the initial choices made by each one of them separately. [p 99]
  • In many respects this hypothesis is the most important one. It indicates how greatly the direction taken by collective opinions and judgements is predetermined, no matter what one does. Thus they are predetermined by the store of previous knowledge and values, and up to a certain point by the collective memory that all share before they meet, and which are all ingrained in them. [p 99]
    • This is the inertia of the group moving across belief (value????) space. And this is why the flocking algorithm, based only on heading and velocity is appropriate.

9:00 – BRI

  • See if GeoMesa example works again
  • Unable to run hdfs dfs -mkdir /hbase/lib because safe mode was on. Went on a disk-cleaning rampage, including removing libre office and got safe mode off
  • And because I was on a roll, installed Bleachbit and gave it a shot
  • Still fails the GeoMesaIngest tests due to a null pointer. Going back to the HBaseQuickStart program.
  • My new IntelliJ needed to download many things
  • Quickstart works! So we’re back to the starting gate. Now why is GeoMesaIngest blowing up?
    org.springframework.web.util.NestedServletException: Request processing failed; nested exception is java.lang.NullPointerException
    
    	at com.vistronix.geomesaingest.web.GeoMesaIngestControllerTest.testIngestJson(GeoMesaIngestControllerTest.java:80)
    Caused by: java.lang.NullPointerException
    	at com.vistronix.geomesaingest.service.GeoMesaIngestor.ingestJsonData(GeoMesaIngestor.java:60)
    	at com.vistronix.geomesaingest.web.GeoMesaIngestController.ingestJson(GeoMesaIngestController.java:44)
    
  • Ok, it turns out that the Mock tests are accessing the actual controller that is in turn accessing the db, which not right at all. Need to rewrite the current tests and add some more, as well as change the schema so that it works with documents and not patients
  • Also need to be able to handle queries? A person would bridge several documents. How should that work? What does the test look like
  • Realized that -1.0 was a bad flag for no geocoordinate. Changed it to 999 as a static final double

Phil 7.18.17

7:00 – 8:00 Research

  • Feeling blue and it’s hard to focus.
  • More C&C
    • We would have, on the one hand, a mimetic sociability, and, on the other, a cathartic sociability. But once more this is an antimony that depends upon circumstances and upon human nature. If we mention here both lines of thinking it is not in order to plead for one rather than the other, but to put both in perspective. [p 92]
    • Ratio decidendi [p 95]
    • At the same time, the more precise the information, the less the field is left free for differences and individual positions. Gradually these are supplanted by collective positions in the consciousness of group members. If the members continue for long enough, the consensus approximates to these values, just as a house under construction does to the architect’s plan. [p 97]
      • I think that I see this as a dimension reduction issue. Too many dimensions, and the distances are too great to come to a consensus, average or extreme. But some dimensions have very little variability, and can be discarded. Some dimensions can be placed outside of the scope of the discussion while others are included by mutual consent. The process of interactive dimension reduction continues until there is enough range of opinion across a sufficiently small set of dimensions that it is possible for a group opinion to emerge. The qualities would be different for different groups – humans, AI, or genes.
  • Chat with Stacy on messenger this morning:
  • Phil, this made me think of your comment about how robots may be changing humans for the worse:
    Opinion | The Trouble With Sex Robots
    Sexual consent is a serious enough problem without bots encouraging men to think it’s unnecessary.
    nytimes.com
    5:30AM
    Thanks – I downloaded and scanned the source paper. I think my basic concern is that ‘power corrupts’, and the moment our machines afford a power dynamic similar to a human one, there are many opportunities for things to go badly.
    7:40AMInteresting. Can a toaster afford a power dynamic (“Make my toast!”)? What about a life-size blow-up female doll? What about a male real doll?
    I think it’s in the eye of the beholder. I have ‘relationships’ with some of my bicycles that are strong enough that I can’t sell them. But an old phone, arguably something that I spend more time with, I can dispose of easily.
    With respect to the toaster, I think there is a continuum? Imagine at one end, a purely mechanical toaster. Push lever, pause, toast pops up.
    On the other end is a human butler, who you ask to make toast. Machines are now traversing that range. It wouldn’t be hard, for example to have Alexa interact with an IoT toaster to make toast on request.
    And Alexa is designed for you to be curt with it.
    I think I heard on NPR that there are researchers looking into how our interactions with voice-activated technology will impact our interactions with human beings, since the same affordance (speech) is being used on entities place in different power dynamics.
    Pulling this around to my research (Because why wouldn’t you?), these sorts of novel inputs may affect how we flock and move through belief space.

9:00 – 5:30 – BRI

  • Generating the message from GeocoderService, but GeoIngestService fails.
  • Going back to my geomesa-tutorials. That fails too: Exception in thread “main” java.io.IOException: java.util.concurrent.ExecutionException: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/.tmp/data/default/geomesa4/d7c3215c807b2ad9bb32555efbec133a/.regioninfo could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and no node(s) are excluded in this operation.
    • It’s possible the node is full. Trying that first
    • Stopping everything and reformatted.
    • Can’t even copy in the GeoMesa jar:
      ~/hadoop-2.7.3$ bin/hadoop fs -put ${GEOMESA_HOME}dist/hbase/geomesa-hbase-distributed-runtime_2.11-1.3.2-SNAPSHOT.jar hdfs://localhost:9000/hbase/lib/
      Picked up JAVA_TOOL_OPTIONS: -Dgeomesa.hbase.coprocessor.path=hdfs://localhost:9000/hbase/lib/geomesa-hbase-distributed-runtime_2.11-1.3.2-SNAPSHOT.jar
      17/07/18 10:55:05 WARN hdfs.DFSClient: DataStreamer Exception
      org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /hbase/lib/geomesa-hbase-distributed-runtime_2.11-1.3.2-SNAPSHOT.jar._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.
      	at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1571)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3107)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3031)
      	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:725)
      	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
      	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
      	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
      	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
      
      	at org.apache.hadoop.ipc.Client.call(Client.java:1475)
      	at org.apache.hadoop.ipc.Client.call(Client.java:1412)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
      	at com.sun.proxy.$Proxy10.addBlock(Unknown Source)
      	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:418)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
      	at com.sun.proxy.$Proxy11.addBlock(Unknown Source)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1455)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1251)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:448)
      put: File /hbase/lib/geomesa-hbase-distributed-runtime_2.11-1.3.2-SNAPSHOT.jar._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1).  There are 0 datanode(s) running and no node(s) are excluded in this operation.
    • Looks like the problem was that you should not format the dfs multiple times. Hadoop gets out of sync. Deleted the /tmp/hadoop-pfeldman directory and files, then reformatted and restarted. That appears to be working.

Phil 7.17.17

Big, tough ride on Saturday. Still tired!

7:00 – 8:00 Research

  • Is this an AI stampede/echo chamber??
    • “There was no reward to sticking to English language,” says Dhruv Batra, visiting research scientist from Georgia Tech at Facebook AI Research (FAIR). As these two agents competed to get the best deal–a very effective bit of AI vs. AI dogfighting researchers have dubbed a “generative adversarial network”–neither was offered any sort of incentive for speaking as a normal person would. So they began to diverge, eventually rearranging legible words into seemingly nonsensical sentences. Supporting Facebook papers
  • More C&C:
  • In general, no one remains entirely passive when faced with what emanates from other people. They are approved of, or argued with in one of those interior dialogues, those silent conversations, in which the group with whom we are communicating is no longer outside us but within us – which is what ‘thinking’ means. Arguments are adopted because they are better formulated, or because we believe we have discovered them ourselves, although we are often repeating, without being aware of it, those we have heard at the time. And when we exclaim ‘I’ve always thought so, but I didn’t dare say it,’ or, ‘I’ve always said so, that’s plain,’ it matters little whether we are sincere or not. It is a cry for recognition by the group.
  • Thus the consensus of the great majority of the groups undergoes a polarization effect. The effect is weak when communication is carried on passively and impersonally, but grows stronger as soon as communication becomes intense and touches people personally. This signifies that the convergence observable in a group depends more on the level of participation and on reciprocal action between its members than on their individual qualities.  [p 87]
    • This also explains why the explore <-> flocking <-> stampede spectrum can be modeled by so few variables (heading, speed, and influence radius), as processed by the agent. This is a personal process with global effects.
  • Thurstone scale  A Thurstone scale has a number of “agree” or “disagree” statements. It is a unidimensional scale to measure attitudes towards people.
    • This also could be a way of determining dimensions that have large ranges as opposed to highly constrained ones
  • It is revealing that a situation in which one has to choose in a personal fashion renders the judgments and choices more extreme, whereas a situation demanding an impersonal choice favors compromise, or almost does. [p 91]
  • We should bear in mind that all this related to the portrait of no one young man in particular. What would happen now if the participants were presented with photographs of familiar people, socially typical, such as workers or intellectuals? Inasmuch as their characteristics stand out more and attitudes towards them are more marked, it might be expected that the results might be more extreme. This was indeed the case. After the group discussion. it turned out that judgments on the characteristics became more extreme, even on the less important ones. [p 91]

8:45 – 5:00 BRI

  • Working out how to fit the location/document message into the GEM and GeoMesa
  • Using a HashMap to dedupe locations from the document
  • Think I got it built. My mistake was thinking that the GEM had an inheritance structure. Next thing is to send the message to GeoMesa
  • Off into the rain!

Phil 7.14.17

7:00 – 8:00 Research

  • Wrote up some notes from meeting with Aaron
  • More C&C
    • The conflicts or differences between members of the group are normally resolved by convergence towards an extreme position. Yet, depending on whether the discussion is public or private, or the dialogue exterior or interior, the convergence will be more, or less, close to that position. In other words, discussion, in its current meaning, depending on whether the individuals involved are active or passive, determines the extent to which the decision will become polarized. [p 81]
      • The book doesn’t cover CMC discussion, but the following two papers appear to perform similar experiments to the Moscovici work
    • Group and computer-mediated discussion effects in risk decision making
      • Managers individually and in 3-person groups made multiattribute risk choices (two investment alternatives, each with multiple outcomes). Two group decisions were reached during face-to-face discussion, and two were reached during (real-time) computer-mediated discussion. In comparison with prediscussion individual preferences, groups’ multiattribute risk choices and attitudes after face-to-face discussion were risk averse for gains and risk seeking for losses, a tendency predicted by prospect theory and consistent with choice shift and other group extremitization research. By contrast, group decisions during computer-mediated discussion did not shift in the direction of prospect theory predictions. The results are consistent with persuasive-arguments theory, in that computer-mediated discussion contained less argumentation than face-to-face discussion. Social decision schemes were used to evaluate alternative assumptions about the group process. A “(prospect-theory) norm-wins” decision scheme described group choice well in the face-to-face discussion condition, but not in the computer-mediated discussion condition. Another decision scheme, first-advocate wins, which described choices well in both face-to-face and computer-mediated discussions, was explored in a discussion of the role of communication in group decision making.
    • Group Polarization and Computer-Mediated Communication
      • Group polarization is the tendency of people to become more extreme in their thinking following group discussion. It may be beneficial to some, but detrimental to other, organizational decisions. This study examines how computer-mediated communication (CMC) may be associated with group polarization. Two laboratory experiments were carried out. The first experiment, conducted in an identified setting, demonstrated that removal of verbal cues might not have reduced social presence sufficiently to impact group polarization, but removal of visual cues might have reduced social presence sufficiently to raise group polarization. Besides confirming the results of the first experiment, the second experiment showed that the provision of anonymity might also have reduced social presence sufficiently to raise group polarization. Analyses of process data from both experiments indicated that the reduction in social presence might have increased group polarization by causing people to generate more novel arguments and engage in more one-upmanship behavior. Collectively, process and outcome data from both experiments reveal how group polarization might be affected by level of social presence. Specifically, group discussion carried out in an unsupported setting or an identified face-to-face CMC setting tends to result in weaker group polarization. Conversely, group discussion conducted in an anonymous face-to-face CMC setting or a dispersed CMC setting (with or without anonymity) tends to lead to stronger group polarization. Implications of these results for further research and practice are provided

8:30 – BRI

  • Added publishers and subscribers to NLP, Gecoder, and Crawl. This is how I think it should work:
  • Publisher (NLP):
    <publishers>
        <publisher id="masterdata-nlp" name="masterdata-nlp" default="true">
            <exchange>eip_exchange</exchange>
            <routingKey>eip.masterdata.nlp</routingKey>
        </publisher>
    </publishers>
  • Subscriber (Geocoder, single channel):
    <subscribers>
        <subscriber name="subscriber-masterdata-nlp">
            <exchange>eip_exchange</exchange>
            <routingKey>eip.masterdata.nlp.#</routingKey>
        </subscriber>
    </subscribers>
  • Subscriber (MDS, two channels):
    <subscribers>
        <subscriber name="subscriber-masterdata">
            <exchange>eip_exchange</exchange>
            <routingKey>eip.masterdata.#</routingKey>
        </subscriber>
        <subscriber name="subscriber-masterdata-nlp">
            <exchange>eip_exchange</exchange>
            <routingKey>eip.masterdata.nlp.#</routingKey>
        </subscriber>
    </subscribers>
  • That seems to be working fine. Now I need to parse out the LOCATION facts. Here’s the loop that gets the document ID and all the locations:
    String idString = event.getId();
    List<Fact> fList = event.getFacts();
    for(Fact f : fList){
        Result r = (Result)(f.getValue());
        Map<String, List<NamedEntity>> neMap = r.getNamedEntities();
        if(neMap.containsKey("LOCATION")) {
            List<NamedEntity> locList = neMap.get("LOCATION");
            System.out.println(locList.toString());
        }
    }
  • Now I need to assemble a message with document ID and all the locations that have lat/longs. I think the way to do this is to build an event that contains a set of geo-coordinate facts.
    • From Matt:
      • you send an Event with the same metadata fields (domain, id, tag, etc…) as the incoming event. Your event has a facts field. In that field you’ll add the Location entity
      • yes. Fact.value is the object and Fact.index is its index
  • Started to build the event in AMQPMessageListener.onMessage():
    Event gcDocEvent = new Event(event.getId(), event.getType(), event.getDomain(), event.getTag(),
            System.currentTimeMillis(), "MessagingService", "event containing geolocation facts",geoFactList.toString(),geoFactList,
            "1.2.0-SNAPSHOT");

Phil 7.13.17

7:00 – 8:00, 4:00 – 6:00 Research

  • Nice (and tasty!) meeting with Wayne
    • Put together outlines of papers by September 16th(?)
      • Precision and recall considered harmful (started, targeted at CHIIR)
      • Gatekeepers and information diversity (IR and UI) (This could be the above paper, or it might be more generalized)
      • Lone wolves, flocks, and stampedes in information space: (human, genetic, and AI)
      • Maps as mediating objects between human and machine knowledge
  • Meeting today with Aaron
    • Using the ProPublica API seems like a good choice for quality data
    • Much discussion on how projections should be chosen and how much bias is introduced, particularly WRT axis choice. My thoughts are that the axis should be the areas that have highest variance, but there may be better options…
  • More C&C
    • Showing the relationship that exists between , on the one hand, the conflict of opinion and differences in information and, on the other , the eyeball confrontations that lead to consensus. [p 79]
      • There was a presentation at Collective Intelligence 2017 that talked about how the ordering of results would affect the ‘quality’ of downloaded  (vs ‘liked’) items. Random ordering (with no visible rating) of results with no rating provided the most consistent results. Ordering based on visible ratings led to first-mover cascades, regardless of ‘quality’. Ratings and order do seem to behave in some ways for proxies for ‘eyeball confrontations’?
    • The mere presence of other people already produces a movement in this direction. It is hardly surprising that, through looking at and listening to them, one becomes a participant in the dialogue, engaging within oneself in one of those imaginary conversations with which we are all familiar. This is sufficient to spark off a ‘fictitious polemic’ with our friends or superiors, in which we argue with them, and which on occasion leads us to modify our attitudes or choices. [p 80]
      • I think forum lurking is an example of this, though we may seek forums where our imaginary conversations are in line with the crowd.
    • Hannah Arendt’s take: The power of judgment rests on a potential agreement with others, and the thinking process which is active in judging something is not, like the thought process of our reasoning, a dialogue between me and myself, but finds itself, always and primarily, even if I am quite alone in making up my mind, in an anticipated communication with others with whom I know I must finally come to some agreement [ p 80]

8:30 – 3:30 BRI

  • Got Hbase working, now working on NLP
  • NLP is running fine, generting locations. Now I need to catch those in GeoCoder and see if I can get coordinates
    • The Geocoder test failed because “lon”: “-122.0850862” != “lon”: “-122.0850861”
  • Adding in <publisher> role for NLP and reading in with MDS and GeoCoder

 

Phil 7.12.17

7:00 – 8:00 Research

  • Continuing C&C
    • The authors of this study therefore varied the degree of cohesion in the groups formed in their laboratory, whose task was to reach an agreement on the risks to be recommended to the fictitious persons of the questionnaire with which we are now very familiar. The results obtained were in conformity with expectations. It turned out that groups having less cohesion recommended daring options, and groups having more cohesion prudent ones. According to the former groups, the fictitious persons would jump at the chance of changing their job and lifestyle; according to the latter groups, they would be content with their present lot. At the same time it was discovered that groups conscious of their cohesion were little subject to tensions and contradictions; this means that their members showed more esteem for their group and desired more strongly to be together than did members of groups possessing less cohesion. They also declared that agreement was reached with their fellows in greater personal intimacy and in a more favourable atmosphere. These are indications that they have done everything to maintain harmony and minimize the differences between them by avoiding factors leading to discord. In short, as the long-standing theory of Festinger (1950) had predicted, cohesion increases the pressure to conform and leads to the search for a compromise in the group[p 73]
      • In reading this, I think that there may be a pattern where large, diverse groups split into progressively smaller, more cohesive groups, each on their own trajectory. An example of this could be the pattern of schism (and to a lesser degree union) in Christianity
    • Clearly, by favouring divergence, and then debate, through the heterogeneous nature of individuals, through their belonging to different professions, through the distance between individual positions, through a lesser cohesiveness in groups or increased trust among their members, consensus is polarized. Moreover, is it not characteristic of such a consensus for common choices not to be decided in advance by a majority rule or compromise, but discovered during adequate discussion? With this as a basis they are rooted in the collectivity as much as in individuals. This is why those who meet together have an interest in not resembling one another. And yet it is true that birds of a feather flock together. All our collective relationships hinge on this paradox. [p 76]
      • Is this a manifestation of explore/exploit? I think so.
    • Thus it is knowledge gleaned from several sources that fuels discussion among them. They are the cornerstones of a well-informed society, a collective organism that is endowed with the power of thought. But the organism shares out among individuals the task of selecting and exploiting the various kinds of knowledge, as well as the job of imparting meaning to words (Putnam, 1979). [p 76]
      • Looks like the authors may think this too
    • In half the groups all their members listened to the proofs in the same order; in the other half, each member listened to them set out in a special order that differed for each member. The first set of information was homogeneous, the second heterogeneous. Moreover, twelve juries listened to proofs that inculpated the accused, and the twelve others to proofs that exculpated him. According to the usual procedure, after listening in court to the facts presented, the jurors assessed separately the degree of guilt of the accused. Then, meeting together as a jury, they discussed the case before evaluating once more separately the degree of guilt. Here we are very close to a real life situation; hence the great significance of the findings.The following is what emerged: consultation together, yet again, led to more decisive verdicts. The difference was even more marked in the groups where each juror heard the proofs in a different order than in those which listened to them in the same order. In other words, when the task of cognition is divided up, the groups polarize more than when the task is uniform. One consequence among others is the following. It is often recommended that jurors should be selected from people whose social origins and intellectual training are as diverse as possible, that is, based on reality, in order to ensure fairer verdicts. The suspicion is that in this way they may be either more clement or more severe. In any case, the analysis of the discussions themselves showed that those jurors who had listened to the proofs presented in a different order mentioned a wider variety of facts than did the others, particularly towards the end of their discussion. [p 77]
      • This is near the core of the Precision and Recall Considered Harmful argument.

9:00 – 4:00 BRI

  • Working on getting all of my pieces working.
  • Got CrawlService running in IntelliJ!
  • Payload is coming in just fine using:
    {
    "query": "Illinois&exactTerms=William Malik&orTerms=police arrest officer charge report",
    "requestId": "IntegrationTestWilliamMalik"
    }
  • However, nothing is going to the MDS or NLP. It looks like no crawl is happening?
  • Found out why. Here’s the exception:
    org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'crawler4jCrawlController' defined in class path resource [com/vistronix/crawlservice/config/CrawlerConfig.class]: 
    Bean instantiation via factory method failed; 
    nested exception is org.springframework.beans.BeanInstantiationException: 
    Failed to instantiate [edu.uci.ics.crawler4j.crawler.CrawlController]: 
    Factory method 'crawler4jCrawlController' threw exception; 
    nested exception is java.lang.Exception: couldn't create the storage folder: /data/crawl/1499883629556 does it already exist ?
  • Created /data/crawl folder with rwx permissions
  • Working on getting MDS working. Hbase is not the default, so the following args have to be added to the VM: -DdataStore.type=hbase -DdataStore.host=localhost -DdataStore.port=2181

Phil 7.11.17

7:00 – 8:00 Research

  • Played around with the lit review section using LMN. Here’s a screenshot of the docs as of yesterday: LMN_screenshot_7.11.17
  • Continuing C&C
    • Thus, as a hypothesis concerning the polarization of groups, it may be concluded that the consensus reached will be the more extreme:
      (a) when individuals participate more directly in the discussions;
      (b) when the differences between them, their knowledge and their opinions are more marked;
      (c) when what is at stake in the discussions is perceived by them as valuable. [p 67]

      • What interests me here is (b). I think that there are several measures of difference that matter
        1. The information distance, as determined by amplitude and variance. There is a difference between agreement about two extreme positions that are broadly based and two narrow positions.
        2. The heading alignment. It is possible to arrive at a position from different directions. Is it easier if the headings are similar?
        3. Velocity. Is there a situation where one piece of information is held fixed and everything else is allowed to change? (e.g. The Leader is always right, though the position is in constant flux [Trump supporters know Trump lies. They just don’t care.])
        4. Exogenous visibility. What does the information horizon look like to the discussants? Do they feel as though they are relatively close or far apart? The VI/Emacs disagreements seem both vast and trivial, depending on framing, for example.
    • Plainly, there is no halo effect on questions that are not included in it (the discussion) [p 69]
      • So in emergent groups, what is the discussion? Or do we look for polarizing behavior and infer the point of discussion from that? I think that this implies axis on a dimension-reduced map that might make sense. 
    • we assumed that where discussion had created tension, shifts in the direction of a consensus should be more frequent. To verify this, the distance was measured between the two individuals whose opinions diverged most before the beginning of the discussion. A distance of 1 meant that these two opinions were separated by one point on a seven-point attitude scale. The opinions of the others, whether identical or not among themselves, were located between those of the two individuals who differed most. In the same way a distance of 6 meant that one of the individuals in the group was located at the favourable pole and another at the unfavourable pole. Thus they were opposites; the opinions of the rest were distributed between these poles. It is here, where conflict was greatest, that the maximum polarization should be recorded. In fact, the shifts towards an extreme consensus turned out to be more frequent in the groups when the gap was more than three points than when it was below that figure. [p 69]
    • The finding was simple: the common choices were much more extreme in the groups of five than in those of four, which themselves were more extreme than those in the groups of three. Moreover, they polarized more when they engaged in discussion among themselves than when they proceeded to a silent exchange of notes [p 70]

9:00 – 4:30 BRC

  • Setting up test instance
    • CrawlService – Cloned
    • NLPService – Cloned
    • MasterDataService – Cloned
    • gtc-test-fixtures – Cloned
    • Added Chrome and Postman
    • Added the rmq dash: sudo rabbitmq-plugins enable rabbitmq_management, which runs at http://localhost:15672 with default login and password of guest
    • MasterDataService runs and is visible here: http://localhost:8890/masterdataservice/events
    • To make things easier to run, added a slew of aliases:
      alias mrmq='sudo rabbitmq-plugins enable rabbitmq_management'
      alias ovpn='cd /home/pfeldman/openVPNconfig && sudo openvpn --config bellrock.ovpn && cd ..'
      alias run_crawl='java -jar /home/pfeldman/IdeaProjects/CrawlService/build/libs/crawlservice.war &'
      alias run_geocoder='java -jar /home/pfeldman/IdeaProjects/GeoCoderService/build/libs/geocoderservice.war &'
      alias run_mds='java -jar /home/pfeldman/IdeaProjects/MasterDataService/build/libs/masterdataservice.war &'
      alias run_nlp='java -jar /home/pfeldman/IdeaProjects/NLPService/build/libs/nlpservice.war &'
      alias startdfs='/home/pfeldman/hadoop-2.7.3/sbin/start-dfs.sh'
      alias starthbase='/home/pfeldman/hbase-1.3.1/bin/start-hbase.sh'
      alias startrmq='service rabbitmq-server start'
      alias stopdfs='/home/pfeldman/hadoop-2.7.3/sbin/stop-dfs.sh'
      alias stophbase='/home/pfeldman/hbase-1.3.1/bin/stop-hbase.sh'
      alias stoprmq='service rabbitmq-server start'
      
    • Have the full stack running, but was unable to get crawlservice running with this payload from scheduling service:

Phil 7.10.17

7:00 – 8:00

  • Social Media and News Sources during the 2017 UK General Election
    • Platforms like Twitter and sources like Wikipedia are important parts of the information diet for many citizens. In this data memo, we analyse Twitter data on bot activity and junk news for a week in the final stages of campaigning of the 2017 UK General Election and also present data on Wikipedia page consultations about those parties and leaders. (1) Content about the Labour Party strongly dominated Twitter traffic in this period. (2) Social media users in the UK shared five links to professional news and information for every one link to junk news. (3) Wikipedia queries have gone from being mostly about the Conservative Party and Prime Minister Theresa May to being mostly about the Labour Party and the Labour leader Jeremy Corbyn. (4) In comparison to the first week of the campaign period, we find that users are sharing slightly better quality news content, that automated accounts are generating more traffic about the election, and that more of the automation uses Labour-related hashtags (though may not be from the Labour Party itself). (5) In comparison to trends in other countries, we find that UK users shared better quality information than that which many US users shared during the 2016 US election, but worse quality news and information than was shared during the French 2017 election.
  • Recognizing safety and liveness
    • Informally, a safety property stipulates that “bad things” do not happen during execution of a program and a liveness property stipulates that “good things” do happen (eventually) (Lamport 1977). Distinguishing between safety and liveness properties is useful because proving that a program satisfies a safety property involves an invariance argument while proving that a program satisfies a liveness property involves a well-foundedness argument. Thus, knowing whether a property is safety or liveness helps when deciding how to prove that the property holds.
      • Read about this in this month’s Communications of the ACM. I wonder if it could be applied to the types of social models I’m building and trying to trace in user data.
  • Continuing C&C
    • In contrast to the consensual form, we can understand that the normalized form, which gives only a subordinate role to some members of the group, creates a certain distance, causing the group not to loom so large in the life and consciousness of individuals, so that in the end it appears strange and abstract. Immediately the participants become detached from one another, and instead of being actors become mere spectators in the discussions. [p 62]
    • Since controversy is in proportion to the participation of members, few conflicts are observed, unless it be in the ranks of the leaders. It is as if individuals tended to minimize their ‘investment’ and their attachment to the collectivity, remaining aloof from intrigues, and, so far as possible, conforming to the opinions and actions that were suggested to them. [p 62]
      • Is this what happens on forums and low-participation social systems like comments?
    • Although the one satisfies the need to participate in a more intense way, and one of which people cannot be deprived for long, the other at least provides a substitute for it. [p 62]
      • This could be another affordance of the system. Some way to grade participation and discussion as a threshold of entering?
    • There can scarcely be any doubt that, by meeting and talking together, a group’s members bring out the values predominant among them, ones to which they are attached. In some way their substance is given shape, so that what we hold in common, but is concealed, becomes manifest. [p 65]
    • …consensual participation probably has the effect of raising the level of collective involvement, whereas normative participation lowers it. One may conclude that the former polarizes the decisions leading to consensus, whereas the latter modifies them. The former causes the members of the group to converge on the pole of values already shared by them before they took part in the decision, and the latter towards the just mean. [p 65]

8:30 – 4:30 BRC

Phil 7.7.17

6:00 – ?? Research

  • Continuing C&C
    • This tension arises less from the content of the argument or the difference that exists between them than because the disagreement manifests itself through someone else who has to be faced up to [p 55]
      • So what are the implications of CMC, where the ‘distance’ can be moderated? The spectrum can run from video chat to text chat to forum, to search results. What’s the sweet, frictionless spot that creates stapedes?
    • Thus, throughout controversies and counter-arguments, which resemble body-blows, the members of the group covertly exert upon one another an influence that emphasizes what can draw them closer. Between them can be observed a synchronized, imitative process which transforms every word into a signal, every gesture into a model, and every piece of information into an argument. All the forms of the rhetoric of mind and body become maneuvers through which the distances between the participants grow smaller and the frictions between them are deadened. [p 55]
    • But most frequently, by the very fact of being called upon to discuss, each individual feels himself to be an actor in the ritual and a member of the group instituting it. In this way, group cohesion is reinforced at regular intervals. [p57]
    • Numerous studies justify the assertion that people are more disposed to start out on that painful intellectual and affective path when they have to deal with opposing arguments coming from several sources rather than from one source alone. It is as if a group speaking with several voices were more conspicuous and offered greater room for maneuver than a group with only one voice.
      • Higher dimensions == less constraint?

8:30 – 4:30 BRI

  • Continuing GeoMesa with Hbase. Notes are here