Phil 12.16.15

7:00 – 9:00, 10:30 – 4:30 VTX

  • Since I’ll be missing the scrum today, sent Aaron an email with status. Which is basically until I know if we’re going to have a semantic network for our derived data, I don’t know how to do a taxonomy.
  • Got RabbitMQ running, following the Local RabbitMQ Setup in Confluence. To open a command prompt as full admin, you have to run it from the ‘start’ input field with Ctrl-shift-enter
  • Running the NLPService with errors. Doesn’t seem to be a permissions issue. Sent Balaji an email, but here are the errors for future reference:
    2015-12-16 08:25:24.449 ERROR 3588 --- [pool-8-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_NLPSERVICE/PFELDMAN-NCS - was unable to sen
     heartbeat!
    
    com.sun.jersey.api.client.ClientHandlerException: org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:8761 refused
            at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:184)
            at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:120)
            at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:28)
            at com.sun.jersey.api.client.Client.handle(Client.java:648)
            at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
            at com.sun.jersey.api.client.WebResource.put(WebResource.java:211)
            at com.netflix.discovery.DiscoveryClient.makeRemoteCall(DiscoveryClient.java:1097)
            at com.netflix.discovery.DiscoveryClient.makeRemoteCall(DiscoveryClient.java:1060)
            at com.netflix.discovery.DiscoveryClient.access$500(DiscoveryClient.java:105)
            at com.netflix.discovery.DiscoveryClient$HeartbeatThread.run(DiscoveryClient.java:1583)
            at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
            at java.util.concurrent.FutureTask.run(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            at java.lang.Thread.run(Unknown Source)
    Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:8761 refused
            at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
            at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:151)
            at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:125)
            at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640)
            at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
            at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
            at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:827)
            at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:170)
            ... 14 common frames omitted
    Caused by: java.net.ConnectException: Connection refused: connect
            at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
            at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
            at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
            at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
            at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
            at java.net.PlainSocketImpl.connect(Unknown Source)
            at java.net.SocksSocketImpl.connect(Unknown Source)
            at java.net.Socket.connect(Unknown Source)
            at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
            at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
            ... 21 common frames omitted
    
    
    2015-12-16 08:25:26,620 ERROR [pool-9-thread-1] com.netflix.discovery.DiscoveryClient [nlp-service-local] Can't get a response from http://localhost:8761/eurek
    /apps/
    Can't contact any eureka nodes - possibly a security group issue?
    com.sun.jersey.api.client.ClientHandlerException: org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:8761 refused
            at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:184) ~[jersey-apache-client4-1.11.jar!/:1.11]
            at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:120) ~[jersey-client-1.11.jar!/:1.11]
            at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:28) ~[eureka-client-1.1.147.jar!/:1.1.147]
            at com.sun.jersey.api.client.Client.handle(Client.java:648) ~[jersey-client-1.11.jar!/:1.11]
            at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670) ~[jersey-client-1.11.jar!/:1.11]
            at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) ~[jersey-client-1.11.jar!/:1.11]
            at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:503) ~[jersey-client-1.11.jar!/:1.11]
            at com.netflix.discovery.DiscoveryClient.getUrl(DiscoveryClient.java:1567) [eureka-client-1.1.147.jar!/:1.1.147]
            at com.netflix.discovery.DiscoveryClient.makeRemoteCall(DiscoveryClient.java:1113) [eureka-client-1.1.147.jar!/:1.1.147]
            at com.netflix.discovery.DiscoveryClient.makeRemoteCall(DiscoveryClient.java:1060) [eureka-client-1.1.147.jar!/:1.1.147]
            at com.netflix.discovery.DiscoveryClient.getAndStoreFullRegistry(DiscoveryClient.java:835) [eureka-client-1.1.147.jar!/:1.1.147]
            at com.netflix.discovery.DiscoveryClient.fetchRegistry(DiscoveryClient.java:746) [eureka-client-1.1.147.jar!/:1.1.147]
            at com.netflix.discovery.DiscoveryClient.access$1400(DiscoveryClient.java:105) [eureka-client-1.1.147.jar!/:1.1.147]
            at com.netflix.discovery.DiscoveryClient$CacheRefreshThread.run(DiscoveryClient.java:1723) [eureka-client-1.1.147.jar!/:1.1.147]
            at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [na:1.8.0_66]
            at java.util.concurrent.FutureTask.run(Unknown Source) [na:1.8.0_66]
            at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.8.0_66]
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.8.0_66]
            at java.lang.Thread.run(Unknown Source) [na:1.8.0_66]
    Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:8761 refused
            at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190) ~[httpclient-4.2.1.jar!/:4.2.1]
            at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:151) ~[httpclient-4.2.1.jar!/:4.2.1]
            at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:125) ~[httpclient-4.2.1.jar!/:4.2.1]
            at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640) ~[httpclient-4.2.1.jar!/:4.2.1]
            at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479) ~[httpclient-4.2.1.jar!/:4.2.1]
            at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) ~[httpclient-4.2.1.jar!/:4.2.1]
            at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:827) ~[httpclient-4.2.1.jar!/:4.2.1]
            at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:170) ~[jersey-apache-client4-1.11.jar!/:1.11]
            ... 18 common frames omitted
    Caused by: java.net.ConnectException: Connection refused: connect
            at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_66]
            at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source) ~[na:1.8.0_66]
            at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) ~[na:1.8.0_66]
            at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) ~[na:1.8.0_66]
            at java.net.AbstractPlainSocketImpl.connect(Unknown Source) ~[na:1.8.0_66]
            at java.net.PlainSocketImpl.connect(Unknown Source) ~[na:1.8.0_66]
            at java.net.SocksSocketImpl.connect(Unknown Source) ~[na:1.8.0_66]
            at java.net.Socket.connect(Unknown Source) ~[na:1.8.0_66]
            at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) ~[httpclient-4.2.1.jar!/:4.2.1]
            at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) ~[httpclient-4.2.1.jar!/:4.2.1]
            ... 25 common frames omitted
    
    2015-12-16 08:25:26.652 ERROR 3588 --- [pool-9-thread-1] com.netflix.discovery.DiscoveryClient    : DiscoveryClient_NLPSERVICE/PFELDMAN-NCS - was unable to ref
    esh its cache! status = org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:8761 refused
    
    com.sun.jersey.api.client.ClientHandlerException: org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:8761 refused
            at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:184)
            at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:120)
            at com.netflix.discovery.EurekaIdentityHeaderFilter.handle(EurekaIdentityHeaderFilter.java:28)
            at com.sun.jersey.api.client.Client.handle(Client.java:648)
            at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
            at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
            at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:503)
            at com.netflix.discovery.DiscoveryClient.getUrl(DiscoveryClient.java:1567)
            at com.netflix.discovery.DiscoveryClient.makeRemoteCall(DiscoveryClient.java:1113)
            at com.netflix.discovery.DiscoveryClient.makeRemoteCall(DiscoveryClient.java:1060)
            at com.netflix.discovery.DiscoveryClient.getAndStoreFullRegistry(DiscoveryClient.java:835)
            at com.netflix.discovery.DiscoveryClient.fetchRegistry(DiscoveryClient.java:746)
            at com.netflix.discovery.DiscoveryClient.access$1400(DiscoveryClient.java:105)
            at com.netflix.discovery.DiscoveryClient$CacheRefreshThread.run(DiscoveryClient.java:1723)
            at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
            at java.util.concurrent.FutureTask.run(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
            at java.lang.Thread.run(Unknown Source)
    Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:8761 refused
            at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
            at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:151)
            at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:125)
            at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:640)
            at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
            at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
            at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:827)
            at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:170)
            ... 18 common frames omitted
    Caused by: java.net.ConnectException: Connection refused: connect
            at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
            at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
            at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
            at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
            at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
            at java.net.PlainSocketImpl.connect(Unknown Source)
            at java.net.SocksSocketImpl.connect(Unknown Source)
            at java.net.Socket.connect(Unknown Source)
            at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
            at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
            ... 25 common frames omitted
  • It turns out that these errors are related to service registering with the discovery process. In local environment you don’t have a service registry running. You can run it if you want and that’s a different process(project). Irrespective of those service trying to register errors, the REST calls will still work.
    • The way to remove the service registration (remove the noise errors) in local env is, in src/main/resources/nlpservice-config.xml, change this
    • <serviceRegistry>http://localhost:8761/eureka/</serviceRegistry&gt;
    • to
    • <serviceRegistry>none</serviceRegistry>
    • Do the gradle build again and run it.
    • And a useful thread:
      Patakula, Balaji 11:44a 
      Hello Phil
      
      Me 11:44a 
      Hiya
      
      Patakula, Balaji 11:45a 
      the url should be localhost:8870/nlpservice/ner
      
      Patakula, Balaji 11:45a 
      with no double slash after the port
      
      Me 11:46a 
      localhost:8870/nlpservice/ner gives the same error in my setup
      
      Patakula, Balaji 11:46a 
      also the body should be like { "text": "my name is Phil Feldman."}
      
      Patakula, Balaji 11:46a 
      or any text that u want
      
      Me 11:47a 
      So it's not the JSON object on the NLPService page?
      
      
      Patakula, Balaji 11:48a 
      if u import the nlp.json into the postman
      
      Patakula, Balaji 11:48a 
      all the requests will be already there
      
      Me 11:49a 
      Import how?
      Patakula, Balaji 11:49a 
      there is an import menu on postman
      
      Me 11:50a 
      Looking for it...
      
      Patakula, Balaji 11:50a 
      the middle panel on the top black menu last item
      
      Me 11:50a 
      Got it.
      
      Patakula, Balaji 11:51a 
      just import that json downloaded from the wiki
      
      Patakula, Balaji 11:51a 
      and u should have the collection now in ostman 
      
      Patakula, Balaji 11:51a 
      postman 
      
      
      
      Patakula, Balaji 11:51a 
      and u can just click to send
      
      Me 11:52a 
      Added the file. Now what.
      
      Patakula, Balaji 11:52a 
      can u share the screen
      
      Me 11:52a 
      using what?
      
      Patakula, Balaji 11:52a 
      u have the nlp service running?
      
      Patakula, Balaji 11:53a 
      just in IM 
      
      Patakula, Balaji 11:53a 
      there is a present screen on the bottom of this chat
      
      Me 11:53a 
      nlp service is running. It extracted my entity as well. Now I'm curious about that bigger json file
      
      Patakula, Balaji 11:54a 
      that json file is just the REST calls that are supported by the service
      
      Patakula, Balaji 11:54a 
      it is just a way of documenting the REST 
      
      Patakula, Balaji 11:54a 
      so some one can just import the file and execute the commands
      
      Me 11:54a 
      So how does it get ingested?
      
      Patakula, Balaji 11:55a 
      which one?
      
      Me 11:55a 
      nlp.json
      
      Patakula, Balaji 11:55a 
      its not ingested. NLP is service. It gets the requests through Rabbit queue from the Crawler ( another service)
      
      Patakula, Balaji 11:56a 
      if u need to test the functionality of NLP, the way u can test and see the results is using the REST interface that we are doing now
      
      Me 11:56a 
      so nlp.json is a configuration file for postman?
      
      Patakula, Balaji 11:56a 
      thats right
      
      Me 11:57a 
      Ah. Not obvious.
      
      Patakula, Balaji 11:57a 
      Is Aaron sit next to you?
      
      Me 11:58a 
      No, he stepped out for a moment. He should be back in 30 minutes or so.
      
      Patakula, Balaji 11:58a 
      may be u can get the data flow from him and he knows how to work with all these tools
      
      Me 11:58a 
      Yeah, he introduced me to Postman.
      
      Me 11:58a 
      But he thought nlp.json was something to send to the NLPService.
      
      Patakula, Balaji 11:58a 
      may be he can give a brain dump of the stuff and how services interact, how data flows etc.,
      
      Me 11:59a 
      I'm starting to see how it works. Was not expecting to see Erlang.
      
      Me 12:00p 
      Can RabbitMQ coordinate services under development on my machine with services stood up on a test environment, such as AWS?
      
      Patakula, Balaji 12:01p 
      u can document all the REST calls that a service exposes by hand writing all those ...or just export the REST calls from postman and every one who wants to use the service can just import that json and work with the REST interface
      
      Me 12:01p 
      Got it.
      
      Patakula, Balaji 12:01p 
      RabbitMq is written in Erlang and we interface with it for messaging
      
      Patakula, Balaji 12:02p 
      yes, u can configure the routes to work that way
      
      Patakula, Balaji 12:02p 
      meaning mismatch services between different environments
      
      Me 12:02p 
      Yeah, I see that. Not that surprising that a communications manager would be written in Erlang. But still a rare thing to see.
      
      Me 12:03p 
      Is there a collection of services stood up that way for development?
      
      Patakula, Balaji 12:04p 
      u installed rabbit yesterday locally on ur machine
      
      Me 12:04p 
      Yes, otherwise none of this would be working?
      
      Patakula, Balaji 12:04p 
      so u can run various services now orchestrated through ur local rabbit
      
      
      
      Me 12:05p 
      Understood. Are there currently stood-up services that can be accessed on an ad-hoc basis, or would I need to do that?
      
      Patakula, Balaji 12:05p 
      Rabbit is only for streaming messages. Every service exposes both streaming ( Rabbitmq messages) and REST interfaces
      
      Patakula, Balaji 12:06p 
      there are no services stood up in adhoc env currently. There is a CI ,QA and Demo env
      
      Patakula, Balaji 12:06p 
      all those envs have all the services running
      
      Me 12:07p 
      What's Cl?
      
      Me 12:07p 
      I'd guess continuous integration, but it's ambiguous.
      
      Patakula, Balaji 12:07p 
      continuous integration. Every code checkin automatically builds the system, runs the tests, creates docker images and deploys those services and starts them 
      
      Me 12:08p 
      Can these CI services be pinged directly?
      
      Patakula, Balaji 12:08p 
      ye
      
      Patakula, Balaji 12:08p 
      yes
      
      Me 12:09p 
      Do you need to be on the VPN?
      
      Patakula, Balaji 12:09p 
      http://dockerapps.philfeldman.com:8763/ <http://dockerapps.philfeldman.com:8763/>  
      
      Patakula, Balaji 12:09p 
      those are the services running
      
      Patakula, Balaji 12:09p 
      and dockerapps is the host machine for CI
      
      Me 12:09p 
      And how do I access the NLPService on dockerapps?
      
      Patakula, Balaji 12:10p 
      access meaning? u want t send the REST requests to CI service?
      
      Me 12:10p 
      Yeah. Bad form?
      
      Patakula, Balaji 12:11p 
      just in the REST, change the localhost to dockerapps.philfeldman.com
      
      Me 12:12p 
      I get a 'Could not get any response'
      
      Me 12:12p 
      dockerapps.philfeldman.com:8870/nlpservice/ner
      
      Patakula, Balaji 12:12p 
      sorry, NLP is running on a different host 10.18.7.177
      
  • Learning about RabbitMQ
  • Installing the google chrome Postman plugin
    • Set the POST option
    • Set RAW
    • Header to Content-Type
    • Value to application/json
    • URL is localhost:8870//nlpservice/ner
    • place the JSON in the ‘Body’ tag
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: