Quantcast
Channel: A Portal to a Portal
Viewing all 1851 articles
Browse latest View live

Further stories of a tinkerer - this time it's IBM BPM, Apache Kibana and Elasticsearch

$
0
0
As per my previous post: -


I'm using this: -


as inspiration.

Start the Deployment Environment

/opt/ibm/WebSphere/AppServer/bin/BPMConfig.sh -start -profile Dmgr01 -de De1

Enable BPM Analytics

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython -f /opt/ibm/WebSphere/AppServer/BPM/Lombardi/tools/def/EnableBPMAnalytics.py

Realm/Cell Name: <default>
Username: wasadmin
Password:         
 WASX7209I: Connected to process "dmgr" on node Dmgr using SOAP connector;  The type of process is: DeploymentManager
JMS Queue Connection Factory "monitorCF" in cell scope is created.
Queue destination "monitorDestination" on bus "BPM.De1.Bus" is created.
JMS Queue "monitorQueue" in cell scope is created.
JMS Activation specification "defAS" in cell scope is created.
Native JSON DEF event is configured. defListenerId is "jmsListenerForJSON".
DEF is reloaded.
WASX7327I: Contents of was.policy file:
 //
// Template policy file for enterprise application.
// Extra permissions can be added if required by the enterprise application.
//
// NOTE: Syntax errors in the policy files will cause the enterprise application FAIL to start.
//       Extreme care should be taken when editing these policy files. It is advised to use
//       the policytool provided by the JDK for editing the policy files
//       (WAS_HOME/java/jre/bin/policytool). 
//

grant codeBase "file:${application}" {
};

grant codeBase "file:${jars}" {
};

grant codeBase "file:${connectorComponent}" {
};

grant codeBase "file:${webComponent}" {
};

grant codeBase "file:${ejbComponent}" {
};


ADMA5016I: Installation of BPMEventEmitter_war_De1 started.
ADMA5058I: Application and module versions are validated with versions of deployment targets.
ADMA5005I: The application BPMEventEmitter_war_De1 is configured in the WebSphere Application Server repository.
ADMA5005I: The application BPMEventEmitter_war_De1 is configured in the WebSphere Application Server repository.
ADMA5081I: The bootstrap address for client module is configured in the WebSphere Application Server repository.
ADMA5053I: The library references for the installed optional package are created.
ADMA5005I: The application BPMEventEmitter_war_De1 is configured in the WebSphere Application Server repository.
ADMA5001I: The application binaries are saved in /opt/ibm/WebSphereProfiles/Dmgr01/wstemp/Script15f0620c96e/workspace/cells/PCCell1/applications/BPMEventEmitter_war_De1.ear/BPMEventEmitter_war_De1.ear
ADMA5005I: The application BPMEventEmitter_war_De1 is configured in the WebSphere Application Server repository.
SECJ0400I: Successfully updated the application BPMEventEmitter_war_De1 with the appContextIDForSecurity information.
ADMA5005I: The application BPMEventEmitter_war_De1 is configured in the WebSphere Application Server repository.
ADMA5005I: The application BPMEventEmitter_war_De1 is configured in the WebSphere Application Server repository.
ADMA5113I: Activation plan created successfully.
ADMA5011I: The cleanup of the temp directory for application BPMEventEmitter_war_De1 is complete.
ADMA5013I: Application BPMEventEmitter_war_De1 installed successfully.
Saving Configuration


Stop the Deployment Environment

/opt/ibm/WebSphere/AppServer/bin/BPMConfig.sh -stop -profile Dmgr01 -de De1

Start the Deployment Environment

/opt/ibm/WebSphere/AppServer/bin/BPMConfig.sh -start -profile Dmgr01 -de De1

Navigate to the Process Portal


Launch a BPD

I chose Standard HR Open New Position - other processes are available, though :-)

Navigate to the Integrated Solutions Console



Edit the BPM Event Emitter YAML file

-I'm using a standard 3-cluster Deployment Environment, so this file is for the Support Cluster

vi /opt/ibm/WebSphereProfiles/Dmgr01/config/cells/PCCell1/clusters/SupCluster/analytics/config/BPMEventEmitter.yml

I only needed to make one change: -

# Configuration properties for the Elasticsearch server
# Elasticsearch is the default event consumer.
# The monitor event is transformed into the Kibana query-optimized format
# before it is written to the Elasticsearch index as a document.
esConfiguration:
    hosts: macbook:9200
    enabled: true
    # The Elasticsearch index name
    index: bpm-events
    # Enable the following properties when Elasticsearch security is on.
    username: elastic
    password: <xor>d3hrdXJEeXU=
    httpsTrustType:
    trustFileLocation:
    hostnameVerifier:
    esTaskIndex: restore_task_index

# The identifier for this BPM environment
# It can be the cell name or any other appropriate identifier.     
bpmCellName: bpmCell01


as I'm NOT using security within Elasticsearch, which is running in a Docker container on my MacBook.

For the record, I validated that the VM hosting BPM could access Elasticsearch: -


{
  "name" : "i1VZYw9",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Dc8j18ioSpWV1Qm16F0pcA",
  "version" : {
    "number" : "5.6.2",
    "build_hash" : "57e20f3",
    "build_date" : "2017-09-23T13:16:45.703Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.1"
  },
  "tagline" : "You Know, for Search"
}

Fully Synchronise the Nodes

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython -f fullSync.jy

Start the Application

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython

AdminControl.invoke('WebSphere:name=ApplicationManager,process=SupClusterMember1,platform=proxy,node=Node1,version=8.5.5.12,type=ApplicationManager,mbeanIdentifier=ApplicationManager,cell=PCCell1,spec=1.0', 'startApplication', '[BPMEventEmitter_war_De1]')

Check the logs

tail -f /opt/ibm/WebSphereProfiles/AppSrv01/logs/SupClusterMember1/SystemOut.log

...
[10/10/17 18:09:46:094 BST] 00000077 ConfigConnect I   CWMCD1002I: The Kafka connection is disabled.
[10/10/17 18:09:46:103 BST] 00000077 RestClientMan W   CWMCD0030W: The current Elasticsearch channel for localhost:9200 is not secured. IBM highly recommends that you secure the channel to prevent network attacks. Refer to the user's documentation to enable security.
[10/10/17 18:09:46:198 BST] 00000077 ConfigConnect I   CWMCD1005I: The ElasticSearch connection is created.
[10/10/17 18:09:46:230 BST] 00000077 LifeCycleMana E class com.ibm.bpm.mon.oi.LifeCycleManageBean startMethod CWMCD1016E: The creation of the Elasticsearch index mapping failed on an exception.
                                 java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:731)
        at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:171)
        at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:145)
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348)
        at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192)
        at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
        at java.lang.Thread.run(Thread.java:785)

[10/10/17 18:09:46:234 BST] 00000077 BusinessExcep E   CNTR0020E: EJB threw an unexpected (non-declared) exception during invocation of method "startMethod" on bean "BeanId(BPMEventEmitter_war_De1#BPMEventEmitter.war#LifeCycleManageBean, null)". Exception data: java.lang.RuntimeException: CWMCD1016E: The creation of the Elasticsearch index mapping failed on an exception.
        at com.ibm.bpm.mon.oi.LifeCycleManageBean.startMethod(LifeCycleManageBean.java:68)


Stop, have a nice cup of tea and regroup

There WAS a clue in the logs above, but it took me several hours to find it …

Yeah, you've spotted it … why do the logs refer to localhost when I've configured BPM to connect to Elasticsearch via the socket macbook:9200 ?

I'd obviously misread the documentation, specifically this PDF IBM Business Process Manager Analytics which said, in part: -




I had ONLY edited the cluster-level file and not the server-level file.

Once I did this: -

vi /opt/ibm/WebSphereProfiles/Dmgr01/config/cells/PCCell1/nodes/Node1/servers/SupClusterMember1/analytics/config/BPMEventEmitter.yml

# Configuration properties for the Elasticsearch server
# Elasticsearch is the default event consumer.
# The monitor event is transformed into the Kibana query-optimized format
# before it is written to the Elasticsearch index as a document.
esConfiguration:
    hosts: macbook:9200
    enabled: true
    # The Elasticsearch index name
    index: bpm-events
    # Enable the following properties when Elasticsearch security is on.
    username: elastic
    password: <xor>d3hrdXJEeXU=
    httpsTrustType:
    trustFileLocation:
    hostnameVerifier:
    esTaskIndex: restore_task_index

# The identifier for this BPM environment
# It can be the cell name or any other appropriate identifier.     
bpmCellName: bpmCell01


re-synchronised the node: -

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython -f fullSync.jy

started the Application

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython

AdminControl.invoke('WebSphere:name=ApplicationManager,process=SupClusterMember1,platform=proxy,node=Node1,version=8.5.5.12,type=ApplicationManager,mbeanIdentifier=ApplicationManager,cell=PCCell1,spec=1.0', 'startApplication', '[BPMEventEmitter_war_De1]')


and checked the logs

tail -f /opt/ibm/WebSphereProfiles/AppSrv01/logs/SupClusterMember1/SystemOut.log

[10/10/17 18:22:30:751 BST] 000000a7 AdminHelper   A   ADMN1008I: An attempt is made to start the BPMEventEmitter_war_De1 application. (User ID = defaultWIMFileBasedRealm/wasadmin)
[10/10/17 18:22:30:760 BST] 000000a7 CompositionUn A   WSVR0190I: Starting composition unit WebSphere:cuname=BPMEventEmitter_war_De1 in BLA WebSphere:blaname=BPMEventEmitter_war_De1.
[10/10/17 18:22:30:822 BST] 000000a7 ApplicationMg A   WSVR0200I: Starting application: BPMEventEmitter_war_De1
[10/10/17 18:22:30:823 BST] 000000a7 ApplicationMg A   WSVR0204I: Application: BPMEventEmitter_war_De1  Application build level: Unknown
[10/10/17 18:22:32:036 BST] 000000a7 AbstractInjec W   CWNEN0070W: The javax.ws.rs.QueryParam annotation class will not be recognized because it was loaded from the file:/opt/ibm/WebSphereProfiles/AppSrv01/installedApps/PCCell1/BPMEventEmitter_war_De1.ear/BPMEventEmitter.war/WEB-INF/lib/jsr311-api-1.1.1.jar location rather than from a product class loader.
[10/10/17 18:22:32:037 BST] 000000a7 AbstractInjec W   CWNEN0070W: The javax.ws.rs.MatrixParam annotation class will not be recognized because it was loaded from the file:/opt/ibm/WebSphereProfiles/AppSrv01/installedApps/PCCell1/BPMEventEmitter_war_De1.ear/BPMEventEmitter.war/WEB-INF/lib/jsr311-api-1.1.1.jar location rather than from a product class loader.
[10/10/17 18:22:32:038 BST] 000000a7 AbstractInjec W   CWNEN0070W: The javax.ws.rs.core.Context annotation class will not be recognized because it was loaded from the file:/opt/ibm/WebSphereProfiles/AppSrv01/installedApps/PCCell1/BPMEventEmitter_war_De1.ear/BPMEventEmitter.war/WEB-INF/lib/jsr311-api-1.1.1.jar location rather than from a product class loader.
[10/10/17 18:22:32:039 BST] 000000a7 AbstractInjec W   CWNEN0070W: The javax.ws.rs.CookieParam annotation class will not be recognized because it was loaded from the file:/opt/ibm/WebSphereProfiles/AppSrv01/installedApps/PCCell1/BPMEventEmitter_war_De1.ear/BPMEventEmitter.war/WEB-INF/lib/jsr311-api-1.1.1.jar location rather than from a product class loader.
[10/10/17 18:22:32:039 BST] 000000a7 AbstractInjec W   CWNEN0070W: The javax.ws.rs.PathParam annotation class will not be recognized because it was loaded from the file:/opt/ibm/WebSphereProfiles/AppSrv01/installedApps/PCCell1/BPMEventEmitter_war_De1.ear/BPMEventEmitter.war/WEB-INF/lib/jsr311-api-1.1.1.jar location rather than from a product class loader.
[10/10/17 18:22:32:040 BST] 000000a7 AbstractInjec W   CWNEN0070W: The javax.ws.rs.HeaderParam annotation class will not be recognized because it was loaded from the file:/opt/ibm/WebSphereProfiles/AppSrv01/installedApps/PCCell1/BPMEventEmitter_war_De1.ear/BPMEventEmitter.war/WEB-INF/lib/jsr311-api-1.1.1.jar location rather than from a product class loader.
[10/10/17 18:22:32:045 BST] 000000a7 SharedEJBRunt I   WSVR0037I: Starting EJB jar: BPMEventEmitter.war
[10/10/17 18:22:32:104 BST] 000000a7 WASNameSpaceB I   CNTR0167I: The server is binding the com.ibm.bpm.mon.oi.LifeCycleManageBean interface of the LifeCycleManageBean enterprise bean in the BPMEventEmitter.war module of the BPMEventEmitter_war_De1 application.  The binding location is: ejblocal:BPMEventEmitter_war_De1/BPMEventEmitter.war/LifeCycleManageBean#com.ibm.bpm.mon.oi.LifeCycleManageBean
[10/10/17 18:22:32:107 BST] 000000a7 WASNameSpaceB I   CNTR0167I: The server is binding the com.ibm.bpm.mon.oi.LifeCycleManageBean interface of the LifeCycleManageBean enterprise bean in the BPMEventEmitter.war module of the BPMEventEmitter_war_De1 application.  The binding location is: ejblocal:com.ibm.bpm.mon.oi.LifeCycleManageBean
[10/10/17 18:22:32:108 BST] 000000a7 AbstractEJBRu I   CNTR0167I: The server is binding the com.ibm.bpm.mon.oi.LifeCycleManageBean interface of the LifeCycleManageBean enterprise bean in the BPMEventEmitter.war module of the BPMEventEmitter_war_De1 application.  The binding location is: java:global/BPMEventEmitter/LifeCycleManageBean!com.ibm.bpm.mon.oi.LifeCycleManageBean
[10/10/17 18:22:32:110 BST] 000000a7 AbstractEJBRu I   CNTR0180I: The BPMEventEmitterMDB message-driven bean in the BPMEventEmitter.war module of the BPMEventEmitter_war_De1 application is bound to the jms/defAS activation specification.
[10/10/17 18:22:32:117 BST] 000000a7 ActivationSpe I   J2CA0291I: Application BPMEventEmitter_war_De1#BPMEventEmitter.war#BPMEventEmitterMDB has an <activation-config-property>, destinationJndiName, for which there is no corresponding property on the ActivationSpec class jms/defAS(com.ibm.ws.sib.api.jmsra.impl.JmsJcaActivationSpecImpl) of ResourceAdapter cells/PCCell1/resources.xml#J2CResourceAdapter_1507551590072. This property will be ignored. This may have undesirable effects.
[10/10/17 18:22:32:154 BST] 000000a7 SibMessage    I   [:] CWSIV0777I: A connection to messaging engine MECluster.000-BPM.De1.Bus for destination monitorDestination on bus BPM.De1.Bus has been successfully created.
[10/10/17 18:22:32:156 BST] 000000a7 ActivationSpe I   J2CA0523I: The Message Endpoint for ActivationSpec jms/defAS (com.ibm.ws.sib.api.jmsra.impl.JmsJcaActivationSpecImpl) and MDB Application BPMEventEmitter_war_De1#BPMEventEmitter.war#BPMEventEmitterMDB is activated.
[10/10/17 18:22:32:183 BST] 000000a7 SharedEJBRunt I   WSVR0057I: EJB jar started: BPMEventEmitter.war
[10/10/17 18:22:32:184 BST] 000000a7 webapp        I com.ibm.ws.webcontainer.webapp.WebGroupImpl WebGroup SRVE0169I: Loading Web Module: BPMEventEmitter.
[10/10/17 18:22:32:191 BST] 000000a7 WASSessionCor I SessionContextRegistry getSessionContext SESN0176I: Will create a new session context for application key default_hostBPMEventEmitter
[10/10/17 18:22:34:113 BST] 000000a7 webcontainer  I com.ibm.ws.webcontainer.VirtualHostImpl addWebApplication SRVE0250I: Web Module BPMEventEmitter has been bound to default_host[*:9080,*:80,*:9443,*:5060,*:5061,*:443,*:9061,*:9044,*:9062,*:9081,*:9444,*:9045,*:9063,*:9082,*:9445,*:9046].
[10/10/17 18:22:34:136 BST] 000000a7 LifeCycleMana I   Application Name:IBM BPM Analytics
[10/10/17 18:22:34:137 BST] 000000a7 LifeCycleMana I   Application Version:8.6.0.0
[10/10/17 18:22:34:137 BST] 000000a7 LifeCycleMana I   Build Level:20170918_46
[10/10/17 18:22:34:248 BST] 000000a7 ConfigConnect I   CWMCD1002I: The Kafka connection is disabled.
[10/10/17 18:22:34:254 BST] 000000a7 RestClientMan W   CWMCD0030W: The current Elasticsearch channel for macbook:9200 is not secured. IBM highly recommends that you secure the channel to prevent network attacks. Refer to the user's documentation to enable security.
[10/10/17 18:22:34:327 BST] 000000a7 ConfigConnect I   CWMCD1005I: The ElasticSearch connection is created.
[10/10/17 18:22:36:780 BST] 000000a7 LifeCycleMana I   CWMCD1014I: The BPMEventEmitter message-driven bean (MDB) is started.
[10/10/17 18:22:36:792 BST] 000000a7 ApplicationMg A   WSVR0221I: Application started: BPMEventEmitter_war_De1
[10/10/17 18:22:36:793 BST] 000000a7 CompositionUn A   WSVR0191I: Composition unit WebSphere:cuname=BPMEventEmitter_war_De1 in BLA WebSphere:blaname=BPMEventEmitter_war_De1 started.

I then checked Kibana, which is running within the same Docker container as Elasticsearch: -


clicked the Play buttton

and, c'est voila, there are my events: -



etc.

I went one stage further and clicked the spanner ( wrench ) icon: -


to get a cURL test client: -

curl -XGET "http://localhost:9200/_search" -H 'Content-Type: application/json' -d'
{
  "query": {
    "match_all": {}
  }
}'

amended it to replace localhost with macbook, and ran it: -

curl -XGET "http://macbook:9200/_search" -H 'Content-Type: application/json' -d'
{
  "query": {
    "match_all": {}
  }
}'


which returned: -

{"took":2,"timed_out":false,"_shards":{"total":11,"successful":11,"skipped":0,"failed":0},"hits":{"total":13,"max_score":1.0,"hits":[{"_index":".kibana","_type":"config","_id":"5.6.2","_score":1.0,"_source":{"buildNum":15543}},{"_index":"bpm-events","_type":"ActivityEvent","_id":"f07332cb-a777-4215-bc49-b97c7c4af9fc_T7169e00460f51792162189","_score":1.0,"_source":{"mon:id":"T7169e00460f51792162189","mon:eventPointData":{"mon:kind":{"mon:version":"2010-11-11","content":"bpmnx:ACTIVITY_ACTIVE"},"mon:time":{"mon:of":"occurrence","content":"2017-10-10T13:26:22.228+01:00"},"ibm:sequenceId":"0000000008","mon:model":[{"mon:id":"bpdid:431b0753c33842e2:3d5457c0:141a2fd3448:-75fb","mon:type":"bpmn:userTask","mon:version":"2064.cca1f786-0c79-4e4e-8539-5767b38c3a48","mon:name":"Submit position request","mon:instance":{"mon:id":"5","mon:role":{"mon:id":"PERFORMER","mon:resource":{"mon:id":"wasadmin","mon:name":"wasadmin"}}},"wle:snapshot-name":"Responsive Hiring Sample v8600_02"},{"mon:id":"c904b3b1-afc1-4698-bf5a-a20892c20275","mon:type":"bpmn:process","mon:version":"2064.cca1f786-0c79-4e4e-8539-5767b38c3a48","mon:name":"Standard HR Open New Position","mon:documentation":"This process covers a manager requesting to create a new position or fill an existing position. The process routes the position request to the Human Resources (HR) department to search for candidates.","mon:instance":{"mon:id":"53","mon:state":"Active"},"wle:snapshot-name":"Responsive Hiring Sample v8600_02"},{"mon:id":"9ab0d0c6-d92c-4355-9ed5-d8a05acdc4b0","mon:type":"wle:processApplication","mon:version":"2064.cca1f786-0c79-4e4e-8539-5767b38c3a48","mon:name":"Hiring Sample","mon:documentation":"Hiring Sample","wle:snapshot-name":"Responsive Hiring Sample v8600_02"}],"mon:correlation":{"mon:ancestor":{"mon:id":"c904b3b1-afc1-4698-bf5a-a20892c20275.2064.cca1f786-0c79-4e4e-8539-5767b38c3a48.53.5"

So that was easy :-)

I've also got the EventSummaryAgent with which to play, but that's another job for another day … :-)

For the record, my IBM colleague, Allen Chan, presented upon this a few months back: -




CWMCB0130E when trying to stop an IBM BPM 8.6 Deployment Environment

$
0
0
I saw this today: -

/opt/ibm/WebSphere/AppServer/bin/BPMConfig.sh -stop -profile Dmgr01 -de De1

Logging to file /opt/ibm/WebSphere/AppServer/logs/config/BPMConfig_20171011-093452.log.
User name of the deployment environment administrator: wasadmin
Password of the deployment environment administrator:          
 
CWMCB0130E: The option is not valid. This option can be used only on the deployment manager machine. Machine is bpm86.uk.ibm.com, install root is /opt/ibm/WebSphere/AppServer.
CWMCB0130E: The option is not valid. This option can be used only on the deployment manager machine. Machine is bpm86.uk.ibm.com, install root is /opt/ibm/WebSphere/AppServer.
The 'BPMConfig.sh -stop -profile Dmgr01 -de De1 -username wasadmin -password ********' command failed. For more information, see the log file /opt/ibm/WebSphere/AppServer/logs/config/BPMConfig_20171011-093452.log.


all of which occurred because the DHCP server embedded within VMware Fusion, in it's infinite wisdom, decided to change the IP address of the VM from 192.168.153.130 to 192.168.153.133.

This was easily fixed by forcing the VM to have a static, rather than dynamic, hostname: -

vi  /etc/sysconfig/network-scripts/ifcfg-eno16777736

changing from: -

TYPE="Ethernet"
BOOTPROTO="dhcp"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_FAILURE_FATAL="no"
NAME="eno16777736"
UUID="1599eaad-7b6c-4d26-b603-16d9aac53fd1"
DEVICE=ens33
ONBOOT="yes"
PEERDNS=yes
PEERROUTES=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes


to: -

TYPE="Ethernet"
BOOTPROTO="static"
IPADDR=192.168.153.130
NETMASK=255.255.255.0
GATEWAY=192.168.153.2

DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_FAILURE_FATAL="no"
NAME="eno16777736"
UUID="1599eaad-7b6c-4d26-b603-16d9aac53fd1"
DEVICE=ens33
ONBOOT="yes"
PEERDNS=yes
PEERROUTES=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes




which I validated thusly: -

ping `hostname`

PING bpm86.uk.ibm.com (192.168.153.130) 56(84) bytes of data.
64 bytes from bpm86.uk.ibm.com (192.168.153.130): icmp_seq=1 ttl=64 time=0.011 ms
64 bytes from bpm86.uk.ibm.com (192.168.153.130): icmp_seq=2 ttl=64 time=0.069 ms
64 bytes from bpm86.uk.ibm.com (192.168.153.130): icmp_seq=3 ttl=64 time=0.034 ms
^C
--- bpm86.uk.ibm.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.011/0.038/0.069/0.023 ms


so now I can do this: -

/opt/ibm/WebSphere/AppServer/bin/BPMConfig.sh -stop -profile Dmgr01 -de De1

Logging to file /opt/ibm/WebSphere/AppServer/logs/config/BPMConfig_20171011-093840.log.
User name of the deployment environment administrator: wasadmin
Password of the deployment environment administrator:          
 
Stopping cluster SupCluster.
Stopping cluster AppCluster.
Stopping cluster MECluster.
The 'BPMConfig.sh -stop -profile Dmgr01 -de De1 -username wasadmin -password ********' command completed successfully.

IBM HTTP Server - Tinkering with CMS Keystore Passwords

$
0
0
Last week, I was demonstrating to a client how one can change the password on a Certificate Management System (CMS) using the IBM Global Security Toolkit (GSK).

Therefore, I'd changed the password from my default ( passw0rd ) to something else ( f00bar ).

To make it 100% clear, this is a sacrificial TEST VM, hence the weak password.

Alas muscle memory makes me type the old the password each and every time ( yes, I can/do use the stashed password when I remember ).

This is how I check what password I currently have: -

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -list -db /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb -pw f00bar

Certificates found
* default, - personal, ! trusted, # secret key
*-wlpn.uk.ibm.com


 and this is how I change it BACK to my favourite ( albeit weak ) password: -

/opt/IBM/HTTPServer/bin/gskcapicmd -keydb -changepw -db /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb -pw f00bar -new_pw passw0rd -stash

-Note that I'm stashing the new password as I change it

and this is how I verify the new password: -

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -list -db /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb -pw passw0rd

Certificates found
* default, - personal, ! trusted, # secret key
*-wlpn.uk.ibm.com


and this is how I verify the new stashed password: -

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -list -db /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb -stashed

Certificates found
* default, - personal, ! trusted, # secret key
*-wlpn.uk.ibm.com


Nice.

Note that I'm using gskcapicmd rather than gskcmd, simply because the former uses a C++ API whereas the latter uses Java, as evidenced below: -

/opt/IBM/HTTPServer/bin/gskcmd -version

iKeyman 8.0.414
CMS provider version 2.57
Java version 1.8.0

(C) Copyright IBM Corp. 2007, 2012.
ALL RIGHTS RESERVED


/opt/IBM/HTTPServer/bin/gskcapicmd -version

GSKCAPICMD
==========
@(#)CompanyName:      IBM Corporation
@(#)LegalTrademarks:  IBM
@(#)FileDescription:  IBM Global Security Toolkit
@(#)FileVersion:      8.0.50.69
@(#)InternalName:     gskcapicmd
@(#)LegalCopyright:   Licensed Materials - Property of IBM GSKit 
                      (C) Copyright IBM Corp.1995, 2016 
                      All Rights Reserved. US Government Users 
                      Restricted Rights - Use, duplication or disclosure
                      restricted by GSA ADP Schedule Contract with IBM Corp.
@(#)OriginalFilename: gsk8capicmd_64
@(#)ProductName:      gsk8j (GoldCoast Build) 160809
@(#)ProductVersion:   8.0.50.69
@(#)ProductInfo:      16/08/03.02:49:36.16/08/09.17:05:03
@(#)CMVCInfo:         gsk8j_160808/gsk8j_doc gsk8j_160808/gsk8j_ikm gsk8j_160808/gsk8j_cms gsk8j_160615/gsk8j_support gsk8j_160525/gsk8j_pkg gsk8j_160803/gsk8j_ssl gsk8j_160413/gsk8j_acme

and it's typically bad practice to have Java installed/running on a web server, especially if it's located within a DMZ, as per this: -



IBM HTTP Server - Checking Personal Certificates

$
0
0
Whilst on the subject of IBM HTTP Server (IHS), as per an earlier post: -


I was "auditing" the SSL certificate that I'm using for IHS, specifically the signature algorithm.

So I have a single certificate in the key store: -

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -list -db /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb -pw passw0rd

Certificates found
* default, - personal, ! trusted, # secret key
*-wlpn.uk.ibm.com


which I validated as follows: -

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -details -db /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb -pw passw0rd -label wlpn.uk.ibm.com

Label : wlpn.uk.ibm.com
Key Size : 2048
Version : X509 V3
Serial : 7554efe3937a2315
Issuer : CN=wlpn.uk.ibm.com
Subject : CN=wlpn.uk.ibm.com
Not Before : 25 June 2017 15:02:51 GMT+01:00

Not After : 26 June 2018 15:02:51 GMT+01:00

Public Key
    30 82 01 22 30 0D 06 09 2A 86 48 86 F7 0D 01 01
    01 05 00 03 82 01 0F 00 30 82 01 0A 02 82 01 01
    00 EA EC 62 79 EE 41 BA 7D CF F1 CD 6E 3E D9 5D
    3E C8 CD F0 3F 04 BA 0E CD EC EA 82 F9 49 74 E6
    9B D4 EB FE B1 8F D0 94 41 F6 33 56 4F 3C AE 3D
    A7 2D 4C 5D 97 19 2A 73 1E 46 11 60 3A 55 37 D1
    BA DD 59 CF 1B 6D 81 B7 F1 DC E5 AB 5B 72 6B A8
    6D D2 C8 8C DF 52 B2 46 90 99 10 93 3B 61 40 46
    94 BF 8B 4B 2E D6 E1 25 78 4E 2F C4 D2 B9 BB A6
    1B DE F2 19 6F 52 0B A7 9B 59 B7 46 65 47 B3 03
    BA B0 DF C7 DA 21 99 CF 4D 82 26 86 89 59 8A 76
    D8 80 21 77 87 95 87 F4 8A 6F C9 2E EF 5B 77 A9
    64 A5 6E 13 16 33 7C 76 7D AC D4 18 FD D7 7A 51
    67 B8 28 D7 32 B2 FA 29 AD 94 9A D6 CD 21 8A 06
    99 3F 38 7A 3E 67 13 6E C2 E9 3D 00 5D 91 74 10
    28 DB 47 56 61 32 BB F3 52 45 0D 0E 4D 30 24 E2
    E0 EC EB 77 13 B8 E5 0D 7D BF BF 1F B7 0A E6 EE
    33 63 C9 AD 3F 44 88 75 AD BC 4A CD 40 85 77 D8
    B7 02 03 01 00 01
Public Key Type : RSA (1.2.840.113549.1.1.1)
Fingerprint : SHA1 : 
    B8 2D BE 82 C8 B7 FF 72 96 10 65 6F 95 09 B4 01
    9B 88 09 C2
Fingerprint : MD5 : 
    65 43 24 27 76 17 2B 11 3B E1 03 FD E4 C8 AC 41
Fingerprint : SHA256 : 
    7F 07 93 14 FE 81 8E 7E 67 16 67 79 0C 68 E4 88
    DB B6 59 2D 62 2D 3B 2B 1B CF 34 EF F6 BD 8B D9
Extensions
    SubjectKeyIdentifier
      keyIdentifier:
    8F CD C5 00 09 45 B6 C5 71 6F B0 92 73 86 23 47
    38 3E A1 82
    AuthorityKeyIdentifier
      keyIdentifier:
    8F CD C5 00 09 45 B6 C5 71 6F B0 92 73 86 23 47
    38 3E A1 82
      authorityIdentifier:
      authorityCertSerialNumber:
Signature Algorithm : SHA1WithRSASignature (1.2.840.113549.1.1.5)
Value
    86 8C 52 F3 92 95 4C 8E C9 A3 E1 46 53 AE 3C 34
    86 FE A7 9D E7 C5 60 E5 C5 99 E5 05 BA FC 03 43
    ED 5D EA C5 15 57 73 73 4C DE 99 A6 A2 89 03 CA
    B4 8B 1C 5B B1 1C 7C 40 B3 EC 4E 18 2B 16 96 15
    9B C0 8E E6 10 FC A3 B6 5A 32 15 B0 B9 AA B4 D8
    C6 48 9E A8 79 24 82 9F 77 44 D4 99 F5 01 AE C2
    84 52 3C 93 32 4C CE 9D 75 7F 7D BD 60 D8 7F E7
    48 12 F2 2C EF 79 76 F6 ED 86 73 00 BE A0 95 B5
    A3 01 6D 0F DD 9A 2C 14 C7 1C B1 79 86 0C E8 71
    22 92 25 5A 8A F5 79 82 9F 05 5A 61 F2 3D 1D 7F
    F2 07 C6 07 A0 21 D4 74 16 F2 F1 96 A0 D6 8C F7
    8E A6 85 BF BC 1E 1C DA DC 09 91 BA 5B A4 00 04
    A9 5E 06 BB 46 78 10 EA 8D 13 5A BE 49 47 FD 7C
    E5 C5 5F 33 76 48 2A CA EB 57 93 2A 73 D8 D2 47
    B4 7A A6 35 5A D7 B6 C6 E5 99 F6 34 81 CD BA BE
    B2 CE C3 9C EF B6 88 62 B5 1C 48 4B 73 6B 48 B9
Trust Status : Enabled


Notice that this has a SHA1 signature algorithm ( SigAlg ), which is potentially less secure than SHA2: -



I also used OpenSSL to check / validate the SigAlg: -

openssl s_client -connect localhost:8443 </dev/null|openssl x509 -text -noout

depth=0 CN = wlpn.uk.ibm.com
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = wlpn.uk.ibm.com
verify return:1
DONE
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 8454646161192133397 (0x7554efe3937a2315)
    Signature Algorithm: sha1WithRSAEncryption
        Issuer: CN=wlpn.uk.ibm.com
        Validity
            Not Before: Jun 25 14:02:51 2017 GMT
            Not After : Jun 26 14:02:51 2018 GMT
        Subject: CN=wlpn.uk.ibm.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:ea:ec:62:79:ee:41:ba:7d:cf:f1:cd:6e:3e:d9:
                    5d:3e:c8:cd:f0:3f:04:ba:0e:cd:ec:ea:82:f9:49:
                    74:e6:9b:d4:eb:fe:b1:8f:d0:94:41:f6:33:56:4f:
                    3c:ae:3d:a7:2d:4c:5d:97:19:2a:73:1e:46:11:60:
                    3a:55:37:d1:ba:dd:59:cf:1b:6d:81:b7:f1:dc:e5:
                    ab:5b:72:6b:a8:6d:d2:c8:8c:df:52:b2:46:90:99:
                    10:93:3b:61:40:46:94:bf:8b:4b:2e:d6:e1:25:78:
                    4e:2f:c4:d2:b9:bb:a6:1b:de:f2:19:6f:52:0b:a7:
                    9b:59:b7:46:65:47:b3:03:ba:b0:df:c7:da:21:99:
                    cf:4d:82:26:86:89:59:8a:76:d8:80:21:77:87:95:
                    87:f4:8a:6f:c9:2e:ef:5b:77:a9:64:a5:6e:13:16:
                    33:7c:76:7d:ac:d4:18:fd:d7:7a:51:67:b8:28:d7:
                    32:b2:fa:29:ad:94:9a:d6:cd:21:8a:06:99:3f:38:
                    7a:3e:67:13:6e:c2:e9:3d:00:5d:91:74:10:28:db:
                    47:56:61:32:bb:f3:52:45:0d:0e:4d:30:24:e2:e0:
                    ec:eb:77:13:b8:e5:0d:7d:bf:bf:1f:b7:0a:e6:ee:
                    33:63:c9:ad:3f:44:88:75:ad:bc:4a:cd:40:85:77:
                    d8:b7
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                8F:CD:C5:00:09:45:B6:C5:71:6F:B0:92:73:86:23:47:38:3E:A1:82
            X509v3 Authority Key Identifier: 
                keyid:8F:CD:C5:00:09:45:B6:C5:71:6F:B0:92:73:86:23:47:38:3E:A1:82

    Signature Algorithm: sha1WithRSAEncryption
         86:8c:52:f3:92:95:4c:8e:c9:a3:e1:46:53:ae:3c:34:86:fe:
         a7:9d:e7:c5:60:e5:c5:99:e5:05:ba:fc:03:43:ed:5d:ea:c5:
         15:57:73:73:4c:de:99:a6:a2:89:03:ca:b4:8b:1c:5b:b1:1c:
         7c:40:b3:ec:4e:18:2b:16:96:15:9b:c0:8e:e6:10:fc:a3:b6:
         5a:32:15:b0:b9:aa:b4:d8:c6:48:9e:a8:79:24:82:9f:77:44:
         d4:99:f5:01:ae:c2:84:52:3c:93:32:4c:ce:9d:75:7f:7d:bd:
         60:d8:7f:e7:48:12:f2:2c:ef:79:76:f6:ed:86:73:00:be:a0:
         95:b5:a3:01:6d:0f:dd:9a:2c:14:c7:1c:b1:79:86:0c:e8:71:
         22:92:25:5a:8a:f5:79:82:9f:05:5a:61:f2:3d:1d:7f:f2:07:
         c6:07:a0:21:d4:74:16:f2:f1:96:a0:d6:8c:f7:8e:a6:85:bf:
         bc:1e:1c:da:dc:09:91:ba:5b:a4:00:04:a9:5e:06:bb:46:78:
         10:ea:8d:13:5a:be:49:47:fd:7c:e5:c5:5f:33:76:48:2a:ca:
         eb:57:93:2a:73:d8:d2:47:b4:7a:a6:35:5a:d7:b6:c6:e5:99:
         f6:34:81:cd:ba:be:b2:ce:c3:9c:ef:b6:88:62:b5:1c:48:4b:
         73:6b:48:b9

Therefore, I wanted to delete and re-issue the self-signed certificate, using a stronger SHA2 SigAlg.

This is what I did: -

Stop IHS

/opt/IBM/HTTPServer/bin/apachectl -k stop -f /opt/IBM/HTTPServer/APIC/conf/httpd.conf

Delete the Self-Signed Certificate

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -delete -db /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb -pw passw0rd -label wlpn.uk.ibm.com

Create a new Self-Signed Certificate

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -create -db /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb -pw passw0rd -sigalg SHA256WithRSA -size 2048 -dn cn=wlpn.uk.ibm.com -san_dnsname wlpn.uk.ibm.com -label wlpn.uk.ibm.com -default_cert yes

-Note that I also included the -san_dnsname parameter to set the Subject Alternate Name (SAN) field - this is to "reassure" Google Chrome that warns against certificates where this field is not set

Validate

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -list -db /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb -pw passw0rd

Certificates found
* default, - personal, ! trusted, # secret key
*-wlpn.uk.ibm.com

/opt/IBM/HTTPServer/bin/gskcapicmd -cert -details -db /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb -pw passw0rd -label wlpn.uk.ibm.com

Label : wlpn.uk.ibm.com
Key Size : 2048
Version : X509 V3
Serial : 7412b8d9509046ab
Issuer : CN=wlpn.uk.ibm.com
Subject : CN=wlpn.uk.ibm.com
Not Before : 10 October 2017 15:42:21 GMT+01:00

Not After : 11 October 2018 15:42:21 GMT+01:00

Public Key
    30 82 01 22 30 0D 06 09 2A 86 48 86 F7 0D 01 01
    01 05 00 03 82 01 0F 00 30 82 01 0A 02 82 01 01
    00 E5 F9 C0 51 4E 56 A3 49 F3 25 29 6C AC 14 12
    F8 9E 6B 4C CE 1B 01 48 A8 63 A3 23 0C CB 4F 31
    8F 1A 57 A1 53 3B 74 1D DF E7 76 2D BA 5C C5 A5
    8D 8F BA C5 8E FC 92 82 89 EE 52 8B F4 B4 47 08
    EE B4 A3 13 3D 73 B8 6D 1E AC 42 A1 E0 DE DB 4C
    65 95 2A EE 9A A8 16 0B FA 49 09 54 28 79 04 7C
    F1 59 62 A3 FA 2B 22 C3 AE 9C 10 73 B6 32 56 27
    36 95 39 2E 9A 6E CD DF EE E2 B6 68 0B D5 D5 59
    BB A3 F4 40 74 22 AA 94 5F 1A 8B D5 15 76 DA 40
    6B C9 F1 13 3B CC B9 FE FA 96 47 C1 D9 BF 91 70
    FF D8 C4 8D 21 ED FC A1 CC 29 84 41 6A 2C 3B 5C
    DF 27 9E 31 84 8F 11 FD FB 81 64 18 8C 46 7E 77
    1A B0 5F 0A 71 B5 B0 7F 80 7D A7 A6 21 94 E5 00
    43 D1 49 1D 36 9F 08 04 4D 36 C0 AE A1 33 27 9D
    6A 3A 3E 20 E8 80 A6 DB 10 2D D5 51 9A FC 69 54
    EE 73 1A 99 17 3E C3 2B 59 CA 30 B5 D4 C4 02 5B
    19 02 03 01 00 01
Public Key Type : RSA (1.2.840.113549.1.1.1)
Fingerprint : SHA1 : 
    C1 DB 70 CF 32 11 96 30 1E 19 62 99 2E B5 C5 A7
    1C B3 6F 3B
Fingerprint : MD5 : 
    CA 9F 04 DA B7 D3 83 31 69 E0 6D 3F C0 6D B2 E8
Fingerprint : SHA256 : 
    75 0D 28 2D FF 87 C8 0B F5 4A 06 31 21 C7 FE 54
    1D 66 E5 26 6C 7B 4C 53 22 1E 98 D8 F8 92 AD 33
Extensions
    subjectAlternativeName
        dNSName: wlpn.uk.ibm.com
    SubjectKeyIdentifier
      keyIdentifier:
    ED 81 80 61 1F 56 F1 B5 97 F2 43 F3 2E 7E A0 CD
    7E 23 F6 92
    AuthorityKeyIdentifier
      keyIdentifier:
    ED 81 80 61 1F 56 F1 B5 97 F2 43 F3 2E 7E A0 CD
    7E 23 F6 92
      authorityIdentifier:
      authorityCertSerialNumber:
Signature Algorithm : SHA256WithRSASignature (1.2.840.113549.1.1.11)
Value
    9B D4 A5 AF DD 3E 29 5D C5 11 81 08 40 3D FF 4B
    1F 1E FA A7 E5 0F C2 9C 95 0A B2 5A F7 B8 29 4E
    3E 3C 85 12 61 3E BB 64 B7 2F 9C 10 AB D3 B6 0C
    B6 56 5C 33 FB 13 A0 CF 3E C3 07 FA 39 66 57 E1
    EE 19 1B 45 F8 DA FE C6 31 F7 57 E9 4B C4 04 69
    8C 9A DE AD CE 88 FF 3D C0 A3 0D 08 3E 21 65 70
    25 2F 68 ED 0D 72 D7 D8 3E 2A C3 D1 61 D7 B2 4C
    75 B4 B2 B1 8D 17 9B E1 D8 F9 C6 05 F3 0E 98 F4
    5A 7C 22 3E 14 C1 68 EE 07 55 DD FF 3F 8E A6 F4
    DE 20 AC B5 E4 59 36 C0 C8 5B 5D 0B A3 4C 5F 63
    5D DC 30 F6 42 30 24 A0 B2 96 A7 BB C8 EB 9D 59
    63 C6 2A 66 E3 B5 D0 56 89 B5 18 F5 8D 3E D9 D1
    1E ED 1A 6E E7 BC C7 71 52 4A 92 C6 A1 64 14 D1
    72 59 F9 9F F2 7F CE 86 03 AF EC 28 74 DC CB D9
    F0 D8 4A 06 13 BC 02 F7 05 7C F2 EF B1 B0 6E 87
    A5 21 7C 7C 75 89 A3 03 CC C5 89 C8 1D C6 76 15
Trust Status : Enabled

Start IHS

/opt/IBM/HTTPServer/bin/apachectl -k start -f /opt/IBM/HTTPServer/APIC/conf/httpd.conf

Validate using OpenSSL

openssl s_client -connect localhost:8443 </dev/null|openssl x509 -text -noout

depth=0 CN = wlpn.uk.ibm.com
verify error:num=18:self signed certificate
verify return:1
depth=0 CN = wlpn.uk.ibm.com
verify return:1
DONE
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 8363950701479478955 (0x7412b8d9509046ab)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=wlpn.uk.ibm.com
        Validity
            Not Before: Oct 10 14:42:21 2017 GMT
            Not After : Oct 11 14:42:21 2018 GMT
        Subject: CN=wlpn.uk.ibm.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:e5:f9:c0:51:4e:56:a3:49:f3:25:29:6c:ac:14:
                    12:f8:9e:6b:4c:ce:1b:01:48:a8:63:a3:23:0c:cb:
                    4f:31:8f:1a:57:a1:53:3b:74:1d:df:e7:76:2d:ba:
                    5c:c5:a5:8d:8f:ba:c5:8e:fc:92:82:89:ee:52:8b:
                    f4:b4:47:08:ee:b4:a3:13:3d:73:b8:6d:1e:ac:42:
                    a1:e0:de:db:4c:65:95:2a:ee:9a:a8:16:0b:fa:49:
                    09:54:28:79:04:7c:f1:59:62:a3:fa:2b:22:c3:ae:
                    9c:10:73:b6:32:56:27:36:95:39:2e:9a:6e:cd:df:
                    ee:e2:b6:68:0b:d5:d5:59:bb:a3:f4:40:74:22:aa:
                    94:5f:1a:8b:d5:15:76:da:40:6b:c9:f1:13:3b:cc:
                    b9:fe:fa:96:47:c1:d9:bf:91:70:ff:d8:c4:8d:21:
                    ed:fc:a1:cc:29:84:41:6a:2c:3b:5c:df:27:9e:31:
                    84:8f:11:fd:fb:81:64:18:8c:46:7e:77:1a:b0:5f:
                    0a:71:b5:b0:7f:80:7d:a7:a6:21:94:e5:00:43:d1:
                    49:1d:36:9f:08:04:4d:36:c0:ae:a1:33:27:9d:6a:
                    3a:3e:20:e8:80:a6:db:10:2d:d5:51:9a:fc:69:54:
                    ee:73:1a:99:17:3e:c3:2b:59:ca:30:b5:d4:c4:02:
                    5b:19
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Alternative Name: 
                DNS:wlpn.uk.ibm.com

            X509v3 Subject Key Identifier: 
                ED:81:80:61:1F:56:F1:B5:97:F2:43:F3:2E:7E:A0:CD:7E:23:F6:92
            X509v3 Authority Key Identifier: 
                keyid:ED:81:80:61:1F:56:F1:B5:97:F2:43:F3:2E:7E:A0:CD:7E:23:F6:92

    Signature Algorithm: sha256WithRSAEncryption
         9b:d4:a5:af:dd:3e:29:5d:c5:11:81:08:40:3d:ff:4b:1f:1e:
         fa:a7:e5:0f:c2:9c:95:0a:b2:5a:f7:b8:29:4e:3e:3c:85:12:
         61:3e:bb:64:b7:2f:9c:10:ab:d3:b6:0c:b6:56:5c:33:fb:13:
         a0:cf:3e:c3:07:fa:39:66:57:e1:ee:19:1b:45:f8:da:fe:c6:
         31:f7:57:e9:4b:c4:04:69:8c:9a:de:ad:ce:88:ff:3d:c0:a3:
         0d:08:3e:21:65:70:25:2f:68:ed:0d:72:d7:d8:3e:2a:c3:d1:
         61:d7:b2:4c:75:b4:b2:b1:8d:17:9b:e1:d8:f9:c6:05:f3:0e:
         98:f4:5a:7c:22:3e:14:c1:68:ee:07:55:dd:ff:3f:8e:a6:f4:
         de:20:ac:b5:e4:59:36:c0:c8:5b:5d:0b:a3:4c:5f:63:5d:dc:
         30:f6:42:30:24:a0:b2:96:a7:bb:c8:eb:9d:59:63:c6:2a:66:
         e3:b5:d0:56:89:b5:18:f5:8d:3e:d9:d1:1e:ed:1a:6e:e7:bc:
         c7:71:52:4a:92:c6:a1:64:14:d1:72:59:f9:9f:f2:7f:ce:86:
         03:af:ec:28:74:dc:cb:d9:f0:d8:4a:06:13:bc:02:f7:05:7c:
         f2:ef:b1:b0:6e:87:a5:21:7c:7c:75:89:a3:03:cc:c5:89:c8:
         1d:c6:76:15

Of course, this won't satisfy some of our more fussy browsers/extensions - SSLSleuth in Firefox only gives us 5.3 out of 10: -


One can mitigate this by reducing the number of ciphers presented by IHS, and restricting it to use ciphers that offer Perfect Forward Secrecy (PFS), as follows: -

LoadModule ibm_ssl_module modules/mod_ibm_ssl.so
Listen 8443
<VirtualHost *:8443>
   SSLProtocolEnable TLSv12
   SSLProtocolDisable SSLv2 SSLv3 TLSv10 TLSv11
   SSLCipherSpec ALL NONE
   SSLCipherSpec TLSv12 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
   SSLCipherSpec TLSv12 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

   SSLEnable
</VirtualHost>
KeyFile /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb
SSLCacheErrorLog /opt/IBM/HTTPServer/APIC/logs/sidd_log
SSLCachePortFilename /opt/IBM/HTTPServer/APIC/logs/siddport
ScriptSock APIC/logs/cgisock
SSLDisable


After a restart, SSLSleuth now gives us a reasonable 9 out of 10: -


and Google Chrome is moderately happy: -


It's major objection, hence the This page is not secure (broken HTTPS) message, is that we're using a self-signed certificate :-(

Final thought - I've mentioned this before, but will mention it again - one can dump out the IHS SSL configuration, and get an summary of what we have: -

/opt/IBM/HTTPServer/bin/apachectl -DDUMP_SSL_CONFIG -f /opt/IBM/HTTPServer/APIC/conf/httpd.conf

SSL configuration:
Default server
Server name: wlpn.uk.ibm.com:0
SSL enabled: NO

SSL server defined at: /opt/IBM/HTTPServer/APIC/conf/httpd.conf:147
Server name: wlpn.uk.ibm.com:8443
SSL enabled: YES
FIPS enabled: 0
Keyfile: /opt/IBM/HTTPServer/APIC/ssl/keystore.kdb
Protocols enabled: TLSv12
Ciphers for SSLV2: (protocol disabled)
Ciphers for SSLV3: (protocol disabled)
Ciphers for TLSv10: (protocol disabled)
Ciphers for TLSv11: (protocol disabled)
Ciphers for TLSv12: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(C02F),TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(C030)

Syntax OK

Job done!


Learning Times - Or Git being a bit of a Git

$
0
0
I'm on a bit of a voyage of discovery this week, having previously been tinkering with Elasticsearch and Kibana a few days back: -

Further stories of a tinkerer - this time it's IBM BPM, Apache Kibana and Elasticsearch

Now I'm playing with Jenkins and Git, the latter of which is running on Docker, with the intention of automating the push of Java code to WebSphere Liberty Profile, also running in Docker.

I also intend to get to grips with Maven at the same time, what's not to like ?

Anyway, back to Git and Jenkins.

I started the Docker container hosting Git as follows: -

git=`docker run -d -p 2222:22 -v ~/git-server/keys:/git-server/keys -v ~/git-server/repos:/git-server/repos jkarlos/git-server-docker`

So I've created a Git repository on a Docker image running on one of my Macs, which is accessible via SSH, and then created a basic Jenkins workflow to pull some Java source ( HelloWorld.java, of course ) from Git, compile it, and execute it.

The next step will then be to push the compiled code to the Liberty container, and execute it. But that's the future …

So this is how I created the Git repository on the remote Mac: -

Make a directory for the myrepo.git repository - note that the path is actually a volume that's used by the Docker container ( see above ) : -

mkdir ~/git-server/repos/myrepo.git

Change to that directory

cd ~/git-server/repos/myrepo.git

Git initialise the repository

git init --shared=true

Add the contents of the current directory to the repository

git add .

Commit the first change

git commit -m "my first commit"

Copy in my source Java from ~/Desktop

cp ~/Desktop/HelloWorld.java .

Add the Java source to the repository

git add HelloWorld.java

Commit the change

git commit -m "Hello World"

Having done this, I then cloned the repository to my main Mac, in order to allow me to work on my Javacode even when remote from the remote Mac: -

Clone the remote repository

Change to the newly created clone repository

cd ~/myrepo

Create a Readme file

touch Readme

Add the Readme file to the remote repository

git add Readme

Edit the Java source

vi HelloWorld.java

Add the updated Java source to the remote repository

git add HelloWorld.java

Commit the changes

git commit -m "Adding changes"

Push the updates

git push

Enter passphrase for key '/Users/davidhay/.ssh/id_rsa': 
Counting objects: 4, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 368 bytes | 368.00 KiB/s, done.
Total 4 (delta 1), reused 0 (delta 0)
remote: error: refusing to update checked out branch: refs/heads/master
remote: error: By default, updating the current branch in a non-bare repository
remote: error: is denied, because it will make the index and work tree inconsistent
remote: error: with what you pushed, and will require 'git reset --hard' to match
remote: error: the work tree to HEAD.
remote: error: 
remote: error: You can set 'receive.denyCurrentBranch' configuration variable to
remote: error: 'ignore' or 'warn' in the remote repository to allow pushing into
remote: error: its current branch; however, this is not recommended unless you
remote: error: arranged to update its work tree to match what you pushed in some
remote: error: other way.
remote: error: 
remote: error: To squelch this message and still keep the default behaviour, set
remote: error: 'receive.denyCurrentBranch' configuration variable to 'refuse'.
To ssh://192.168.1.12:2222/git-server/repos/myrepo.git
 ! [remote rejected] master -> master (branch is currently checked out)
error: failed to push some refs to 'ssh://git@192.168.1.12:2222/git-server/repos/myrepo.git'

At this point, I reverted to normal, and dived into Google for answers.

Eventually, I found this: -

<snip>
This error is showing up because you didn't create the original repo as a bare repository, thus it's "protected" in a way you can't just push to it.

You can do as the error message says and set receive.denyCurrentBranch in the server side, where this repository is originally located. You can do this by navigating to the repository in the original location and running:

git config receive.denyCurrentBranch ignore
</snip>


So, if I understand correctly, when I created the original repository, on the remote Mac, I should've specified that it be "bare" :-)

I tried the circumvention on the remote Mac: -

cd ~/git-server/repos/myrepo.git
git config receive.denyCurrentBranch ignore

and then tried the git push again: -

git push

Enter passphrase for key '/Users/davidhay/.ssh/id_rsa': 
Counting objects: 4, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 368 bytes | 368.00 KiB/s, done.
Total 4 (delta 1), reused 0 (delta 0)
To ssh://192.168.1.12:2222/git-server/repos/myrepo.git
   894abb8..ea975c5  master -> master


This time it worked … the updated files ( Readme and HelloWorld.java ) appear in the remote repository.

Even better than that, my Jenkins workflow works …

In the next post, I will talk about the Jenkins -> remote Git plumbing, a problem that I hit with SSH certificates, and the workflow …

Jenkins to Git - SSH says "No"

$
0
0
As per my earlier post: -


I'm on a voyage of discovery with Jenkins and Git.

Whilst trying to plumb onto into t'other, I was hitting a blocker.

To recap, I have Jenkins installed on my MacBook, running locally, and I have Git running on a Docker container on a remote Mac.

Therefore, I'm connecting to the remote Git repository using SSH rather than, say, HTTPS or a local file-system.

This works OK for me using Git commands such as: -

and: -

git push

So, in the world of Jenkins, I thought it'd be equally simple.

To start with, I created a new job / project: -


chose Git as my SCM: -


added in the SSH URL: -


and immediately saw this: -

Failed to connect to repository : Command "git ls-remote -h ssh://git@192.168.1.214:2222/git-server/repos/myrepo.git HEAD" returned status code 128:
stdout: 
stderr: Permission denied (publickey,keyboard-interactive). 
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

before I'd had a chance to enter some credentials :-(

I clicked the button to add new creds: -


which led me here: -


I tried pasting the private key into the Key field, having used the command: -

ssh-keygen -y -f ~/.ssh/id_rsa

to retrieve the hash that represents that private key.

I'd previously validated that the retrieved hash matches the public key: -

~/.ssh/id_rsa.pub

However, when i switched back to the project configuration screen, I saw this: -

Failed to connect to repository : Command "git ls-remote -h ssh://git@192.168.1.214:2222/git-server/repos/myrepo.git HEAD" returned status code 128:
stdout: 
stderr: Load key "/Users/Shared/Jenkins/tmp/ssh6857222762876740778.key": invalid format 
Permission denied (publickey,keyboard-interactive). 
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

After much faffing about, I switched the Credentials to this: -


i.e. specifically pulling the key from ~/.ssh on the main Mac, upon which Jenkins is running ( hence the Jenkins master ).

But I was still seeing this: -

Failed to connect to repository : Command "git ls-remote -h ssh://git@192.168.1.214:2222/git-server/repos/myrepo.git HEAD" returned status code 128:
stdout: 
stderr: Load key "/Users/Shared/Jenkins/tmp/ssh6850003580465807718.key": invalid format 
Permission denied (publickey,keyboard-interactive). 
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Then I realised that it was looking for the home directory of A DIFFERENT USER :-)

There's even a clue in the error above: -

stderr: Load key "/Users/Shared/Jenkins/tmp/ssh6850003580465807718.key": invalid format 

So I switched to root: -

su -

Changed to the appropriate ~/.ssh directory for the Jenkins user: -

cd /Users/Shared/Jenkins/.ssh

and copied the private key: -

cp /Users/davidhay/.ssh/id_rsa .

and tried again.

Alas: -

Failed to connect to repository : Command "git ls-remote -h ssh://git@192.168.1.214:2222/git-server/repos/myrepo.git HEAD" returned status code 128:
stdout: 
stderr: Load key "/Users/Shared/Jenkins/tmp/ssh2229777690807748085.key": invalid format 
Permission denied (publickey,keyboard-interactive). 
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I validated that the key was in the right place: -

ls -al /Users/Shared/Jenkins/.ssh

total 16
drwx------  4 jenkins  jenkins   128 13 Oct 11:14 .
drwxr-xr-x  7 jenkins  jenkins   224 12 Oct 10:34 ..
-rw-------  1 root     jenkins  1766 13 Oct 11:14 id_rsa
-rw-r--r--  1 jenkins  jenkins   363 13 Oct 10:15 known_hosts


and then noticed the obvious mistake.

Can you see where I went wrong ?

-rw-------  1 root     jenkins  1766 13 Oct 11:14 id_rsa

I changed the permissions: -

chown jenkins:jenkins /Users/Shared/Jenkins/.ssh/id_rsa

validated the change: -

ls -al /Users/Shared/Jenkins/.ssh

total 16
drwx------  4 jenkins  jenkins   128 13 Oct 11:14 .
drwxr-xr-x  7 jenkins  jenkins   224 12 Oct 10:34 ..
-rw-------  1 jenkins  jenkins  1766 13 Oct 11:14 id_rsa
-rw-r--r--  1 jenkins  jenkins   363 13 Oct 10:15 known_hosts

and retried Jenkins: -


To finish, I added a new Build step: -


which compiles and executes the Java sourced from Git, and then ran the Build: -


Whilst I was on the Jenkins master, I also checked the workspace: -

ls -al /Users/Shared/Jenkins/Home/workspace/DaveHay

total 24
drwxr-xr-x   6 jenkins  jenkins  192 13 Oct 10:21 .
drwxr-xr-x   6 jenkins  jenkins  192 12 Oct 13:59 ..
drwxr-xr-x  13 jenkins  jenkins  416 13 Oct 11:26 .git
-rw-r--r--   1 jenkins  jenkins  462 13 Oct 11:26 HelloWorld.class
-rw-r--r--   1 jenkins  jenkins  148 13 Oct 10:21 HelloWorld.java
-rw-r--r--   1 jenkins  jenkins   25 13 Oct 10:18 Readme


which showed the newly compiled Java class.

Job done :-)

Git and Jenkins - Learning Resources

$
0
0

Ubuntu - Software Updater and the Insufficient Disk Space

$
0
0

So I'm trying to update Ubuntu 16.0.4.3 LTS using Software Updater, but couldn't get past this: -

    

Now I have LOADS of disk space: -

df -kmh

Filesystem                   Size  Used Avail Use% Mounted on
udev                          16G     0   16G   0% /dev
tmpfs                        3.2G  9.4M  3.2G   1% /run
/dev/mapper/ubuntu--vg-root  2.7T  346G  2.2T  14% /
tmpfs                         16G  224K   16G   1% /dev/shm
tmpfs                        5.0M  4.0K  5.0M   1% /run/lock
tmpfs                         16G     0   16G   0% /sys/fs/cgroup
/dev/loop1                    81M   81M     0 100% /snap/core/2381
/dev/loop0                    89M   89M     0 100% /snap/conjure-up/527
/dev/loop2                    80M   80M     0 100% /snap/conjure-up/745
/dev/loop3                    81M   81M     0 100% /snap/core/2462
/dev/loop4                    89M   89M     0 100% /snap/conjure-up/549
/dev/loop5                    82M   82M     0 100% /snap/core/2898
/dev/sda2                    473M  363M   86M  81% /boot
/dev/sda1                    511M  3.4M  508M   1% /boot/efi
tmpfs                        3.2G   76K  3.2G   1% /run/user/1000

and yet /boot is 81% full.

A quick Google brought me here: -

Not enough free disk space when upgrading

which had me do this: -

sudo apt-get autoremove

and now I have this: -

df -kmh

Filesystem                   Size  Used Avail Use% Mounted on
udev                          16G     0   16G   0% /dev
tmpfs                        3.2G  9.4M  3.2G   1% /run
/dev/mapper/ubuntu--vg-root  2.7T  344G  2.2T  14% /
tmpfs                         16G   52M   16G   1% /dev/shm
tmpfs                        5.0M  4.0K  5.0M   1% /run/lock
tmpfs                         16G     0   16G   0% /sys/fs/cgroup
/dev/loop0                    89M   89M     0 100% /snap/conjure-up/527
/dev/loop2                    80M   80M     0 100% /snap/conjure-up/745
/dev/loop3                    81M   81M     0 100% /snap/core/2462
/dev/loop4                    89M   89M     0 100% /snap/conjure-up/549
/dev/loop5                    82M   82M     0 100% /snap/core/2898
/dev/sda2                    473M  132M  317M  30% /boot
/dev/sda1                    511M  3.4M  508M   1% /boot/efi
tmpfs                        3.2G   80K  3.2G   1% /run/user/1000
/dev/loop6                    84M   84M     0 100% /snap/core/3017

and now I have this: -

which is nice :-)


IBM Cloud Private - My first foray

$
0
0
So this week, along with many other things, I'm starting to get to grips with the newly announced IBM Cloud Private: -

IBM brings the power of cloud behind the enterprise firewall

I'm running on Ubuntu Linux: -

lsb_release -a

No LSB modules are available.
Distributor ID:    Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:    16.04
Codename:    xenial


so started by installing the pre-requisites of VirtualBox and Vagrant: -

sudo apt-get install virtualbox
sudo apt-get install vagrant

and, having cloned the Git repository: -

https://github.com/IBM/deploy-ibm-cloud-private

I followed the instructions to bring up the Vagrant environment: -

vagrant up

Bringing machine 'icp' up with 'virtualbox' provider...
==> icp: Clearing any previously set forwarded ports...
==> icp: Clearing any previously set network interfaces...
==> icp: Preparing network interfaces based on configuration...
    icp: Adapter 1: nat
    icp: Adapter 2: hostonly
==> icp: Forwarding ports...
    icp: 22 (guest) => 2222 (host) (adapter 1)
==> icp: Running 'pre-boot' VM customizations...
A customization command failed:

["modifyvm", :id, "--apic", "on"]

The following error was experienced:

#<Vagrant::Errors::VBoxManageError: There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

Command: ["modifyvm", "6386ef56-d015-4672-919d-40758eeab63c", "--apic", "on"]

Stderr: Oracle VM VirtualBox Command Line Management Interface Version 5.0.40_Ubuntu
(C) 2005-2017 Oracle Corporation
All rights reserved.

Usage:

VBoxManage modifyvm         <uuid|vmname>
                            [--name <name>]
                            [--groups <group>, ...]
                            [--description <desc>]
                            [--ostype <ostype>]
                            [--iconfile <filename>]
                            [--memory <memorysize in MB>]
                            [--pagefusion on|off]
                            [--vram <vramsize in MB>]
                            [--acpi on|off]
                            [--pciattach 03:04.0]
                            [--pciattach 03:04.0@02:01.0]
                            [--pcidetach 03:04.0]
                            [--ioapic on|off]
                            [--hpet on|off]
                            [--triplefaultreset on|off]
                            [--paravirtprovider none|default|legacy|minimal|
                                                hyperv|kvm]
                            [--hwvirtex on|off]
                            [--nestedpaging on|off]
                            [--largepages on|off]
                            [--vtxvpid on|off]
                            [--vtxux on|off]
                            [--pae on|off]
                            [--longmode on|off]
                            [--cpuid-portability-level <0..3>
                            [--cpuidset <leaf> <eax> <ebx> <ecx> <edx>]
                            [--cpuidremove <leaf>]
                            [--cpuidremoveall]
                            [--hardwareuuid <uuid>]
                            [--cpus <number>]
                            [--cpuhotplug on|off]
                            [--plugcpu <id>]
                            [--unplugcpu <id>]
                            [--cpuexecutioncap <1-100>]
                            [--rtcuseutc on|off]
                            [--graphicscontroller none|vboxvga|vmsvga]
                            [--monitorcount <number>]
                            [--accelerate3d on|off]
                            [--accelerate2dvideo on|off]
                            [--firmware bios|efi|efi32|efi64]
                            [--chipset ich9|piix3]
                            [--bioslogofadein on|off]
                            [--bioslogofadeout on|off]
                            [--bioslogodisplaytime <msec>]
                            [--bioslogoimagepath <imagepath>]
                            [--biosbootmenu disabled|menuonly|messageandmenu]
                            [--biossystemtimeoffset <msec>]
                            [--biospxedebug on|off]
                            [--boot<1-4> none|floppy|dvd|disk|net>]
                            [--nic<1-N> none|null|nat|bridged|intnet|hostonly|
                                        generic|natnetwork]
                            [--nictype<1-N> Am79C970A|Am79C973|
                                            82540EM|82543GC|82545EM|
                                            virtio]
                            [--cableconnected<1-N> on|off]
                            [--nictrace<1-N> on|off]
                            [--nictracefile<1-N> <filename>]
                            [--nicproperty<1-N> name=[value]]
                            [--nicspeed<1-N> <kbps>]
                            [--nicbootprio<1-N> <priority>]
                            [--nicpromisc<1-N> deny|allow-vms|allow-all]
                            [--nicbandwidthgroup<1-N> none|<name>]
                            [--bridgeadapter<1-N> none|<devicename>]
                            [--hostonlyadapter<1-N> none|<devicename>]
                            [--intnet<1-N> <network name>]
                            [--nat-network<1-N> <network name>]
                            [--nicgenericdrv<1-N> <driver>
                            [--natnet<1-N> <network>|default]
                            [--natsettings<1-N> [<mtu>],[<socksnd>],
                                                [<sockrcv>],[<tcpsnd>],
                                                [<tcprcv>]]
                            [--natpf<1-N> [<rulename>],tcp|udp,[<hostip>],
                                          <hostport>,[<guestip>],<guestport>]
                            [--natpf<1-N> delete <rulename>]
                            [--nattftpprefix<1-N> <prefix>]
                            [--nattftpfile<1-N> <file>]
                            [--nattftpserver<1-N> <ip>]
                            [--natbindip<1-N> <ip>
                            [--natdnspassdomain<1-N> on|off]
                            [--natdnsproxy<1-N> on|off]
                            [--natdnshostresolver<1-N> on|off]
                            [--nataliasmode<1-N> default|[log],[proxyonly],
                                                         [sameports]]
                            [--macaddress<1-N> auto|<mac>]
                            [--mouse ps2|usb|usbtablet|usbmultitouch]
                            [--keyboard ps2|usb
                            [--uart<1-N> off|<I/O base> <IRQ>]
                            [--uartmode<1-N> disconnected|
                                             server <pipe>|
                                             client <pipe>|
                                             tcpserver <port>|
                                             tcpclient <hostname:port>|
                                             file <file>|
                                             <devicename>]
                            [--lpt<1-N> off|<I/O base> <IRQ>]
                            [--lptmode<1-N> <devicename>]
                            [--guestmemoryballoon <balloonsize in MB>]
                            [--audio none|null|oss|alsa|pulse]
                            [--audiocontroller ac97|hda|sb16]
                            [--audiocodec stac9700|ad1980|stac9221|sb16]
                            [--clipboard disabled|hosttoguest|guesttohost|
                                         bidirectional]
                            [--draganddrop disabled|hosttoguest]
                            [--vrde on|off]
                            [--vrdeextpack default|<name>
                            [--vrdeproperty <name=[value]>]
                            [--vrdeport <hostport>]
                            [--vrdeaddress <hostip>]
                            [--vrdeauthtype null|external|guest]
                            [--vrdeauthlibrary default|<name>
                            [--vrdemulticon on|off]
                            [--vrdereusecon on|off]
                            [--vrdevideochannel on|off]
                            [--vrdevideochannelquality <percent>]
                            [--usb on|off]
                            [--usbehci on|off]
                            [--usbxhci on|off]
                            [--usbrename <oldname> <newname>]
                            [--snapshotfolder default|<path>]
                            [--teleporter on|off]
                            [--teleporterport <port>]
                            [--teleporteraddress <address|empty>
                            [--teleporterpassword <password>]
                            [--teleporterpasswordfile <file>|stdin]
                            [--tracing-enabled on|off]
                            [--tracing-config <config-string>]
                            [--tracing-allow-vm-access on|off]
                            [--usbcardreader on|off]
                            [--autostart-enabled on|off]
                            [--autostart-delay <seconds>]
                            [--videocap on|off]
                            [--videocapscreens all|<screen ID> [<screen ID> ...]]
                            [--videocapfile <filename>]
                            [--videocapres <width> <height>]
                            [--videocaprate <rate>]
                            [--videocapfps <fps>]
                            [--videocapmaxtime <ms>]
                            [--videocapmaxsize <MB>]
                            [--videocapopts <key=value> [<key=value> ...]]
                            [--defaultfrontend default|<name>]

VBoxManage: error: Unknown option: --apic
>

Please fix this customization and try again.


Suspecting that I'd got the wrong versions of the pre-requisites, I checked what I'd installed: -

vagrant -v

Vagrant 1.8.1

VBoxManage -version

5.0.40_Ubuntur115130

whereas the above Git page specifies: -

Vagrant 2.0.0

VirtualBox 5.1.28

I downloaded the latest versions of both: -

https://www.hashicorp.com/blog/hashicorp-vagrant-2-0/

https://www.virtualbox.org/wiki/Linux_Downloads

and started by installing the new version of Vagrant, and retrying the ICP installation: -

vagrant up

Bringing machine 'icp' up with 'virtualbox' provider...
==> icp: Clearing any previously set forwarded ports...
==> icp: Clearing any previously set network interfaces...
==> icp: Preparing network interfaces based on configuration...
    icp: Adapter 1: nat
    icp: Adapter 2: hostonly
==> icp: Forwarding ports...
    icp: 22 (guest) => 2222 (host) (adapter 1)
==> icp: Running 'pre-boot' VM customizations...
==> icp: Booting VM...
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

Command: ["startvm", "6386ef56-d015-4672-919d-40758eeab63c", "--type", "headless"]

Stderr: VBoxManage: error: The virtual machine 'IBM-Cloud-Private-dev-edition' has terminated unexpectedly during startup with exit code 1 (0x1)
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component MachineWrap, interface IMachine

Assuming that the problem was more with VirtualBox than Vagrant, I installed the new version of that ( which took a bit of work with sudo dpkg --remove and sudo dpkg --purge).

Having validate the versions: -

vagrant -v

Vagrant 2.0.0

VBoxManage -v

5.1.28r117968

This time around: -

vagrant up

Bringing machine 'icp' up with 'virtualbox' provider...
==> icp: Clearing any previously set forwarded ports...
==> icp: Clearing any previously set network interfaces...
==> icp: Preparing network interfaces based on configuration...
    icp: Adapter 1: nat
    icp: Adapter 2: hostonly
==> icp: Forwarding ports...
    icp: 22 (guest) => 2222 (host) (adapter 1)
==> icp: Running 'pre-boot' VM customizations...
==> icp: Booting VM...
==> icp: Waiting for machine to boot. This may take a few minutes...
    icp: SSH address: 127.0.0.1:2222
    icp: SSH username: vagrant
    icp: SSH auth method: private key
==> icp: Machine booted and ready!
==> icp: Checking for guest additions in VM...
==> icp: Setting hostname...
==> icp: Running provisioner: shell...
    icp: Running: script: configure_master_ssh_keys
==> icp: Running provisioner: shell...
    icp: Running: script: configure_swap_space
==> icp: Setting up swapspace version 1, size = 8 GiB (8589930496 bytes)
==> icp: no label, UUID=d5e47d79-2646-4bf8-b89d-45b60ca406ff
==> icp: vm.swappiness = 60
==> icp: vm.vfs_cache_pressure = 10
==> icp: Running provisioner: shell...
    icp: Running: script: configure_performance_settings
==> icp: vm.swappiness = 60
==> icp: vm.vfs_cache_pressure = 10
==> icp: net.ipv4.ip_forward = 1

...

==> icp: Starting cfc-worker2
==> icp: Running provisioner: shell...
    icp: Running: script: wait_for_worker_nodes_to_boot
==> icp:
==> icp: Preparing nodes for IBM Cloud Private community edition cluster installation.
==> icp: This process will take approximately 10-20 minutes depending on network speeds.
==> icp: Take a break and go grab a cup of coffee, we'll keep working on this while you're away ;-)
==> icp: .
==> icp: .
==> icp: .
==> icp: master.icp             ready
==> icp: cfc-worker1.icp         ready
==> icp: cfc-worker2.icp         ready
==> icp: cfc-manager1.icp         ready
==> icp: Running provisioner: shell...
    icp: Running: script: precache_images
==> icp:
==> icp: Seeding IBM Cloud Private installation by pre-caching required docker images.
==> icp: This may take a few minutes depending on your connection speed and reliability.
==> icp: Pre-caching docker images....
==> icp: Pulling ibmcom/icp-inception:2.1.0-beta-3...
==> icp: Pulling ibmcom/icp-datastore:2.1.0-beta-3...
 ==> icp: Pulling ibmcom/icp-platform-auth:2.1.0-beta-3...
 ==> icp: Pulling ibmcom/icp-auth:2.1.0-beta-3...

...

So it hasn't yet finished, but, in the words of Tom Cruise, "It's looking good so far"

:-)

Apple Watch - go, no go, go

$
0
0
So I had a weird experience last evening, and not in a good way.

For no apparent reason, this was my Apple Watch: -


and this: -


I have no earthly idea what happened.

So, being a true nerd, and a big fan of The IT Crowd, I decided to ( all together now ) TURN IT OFF AND ON AGAIN ….

Obviously I couldn't read the display, what with it being all garbled n' all, so I just hit the big button on the right-hand side, below the digital crown and chose the appropriate gibberish - it was the one in red, so it must've been the right one ? Right ?

WRONG !!

The next, my Apple Watch has called 999 ( the UK's emergency services number, similar to 911 in the USA ), and I'm talking to an operator, who's asking how he can help.

When I don't immediately respond ( panic has set in at this point ), he's saying "If you're unable to speak, please press a digit on your phone's dial" etc. assuming, for good reason, that I am injured and cannot respond :-(

I manage to find my voice, and tell him that all is well, and apologise profusely for wasting his time and our public resources ….

Then the house phone rings … and my beloved gets a recorded message telling her that Dave Hay has called the emergency services.

And then I get SMS messages on all my Apple devices …..

And then the home phone rings again, with yet another recorded message with my location ( thanks to Apple Maps ).

In short, the Apple ecosystem has kicked in to save me … even though there's nothing wrong with me, apart from my obvious inability to use Apple hardware.

Finally, I manage to power the watch off, set it on its charging stand, so it can reboot - and all seems well.

For the record, this is what I should've done: -

data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkGBwgHBgkIBwgKCgkLDRYPDQwMDRsUFRAWIB0iIiAdHx8kKDQsJCYxJx8fLT0tMTU3Ojo6Iys/RD84QzQ5OjcBCgoKDQwNGg8PGjclHyU3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3N//AABEIAKAAzwMBIgACEQEDEQH/xAAcAAABBAMBAAAAAAAAAAAAAAAABAUGBwECAwj/xABREAABAgQDAwYHCBAEBwEAAAABAgMABAURBhIhEzFBB1FhcZGSFCIyU4Gh0RUjMzVCUmOyFiUmREZUVVZicnOClLGzwnSiwfA0Q0WEk9LhJP/EABoBAQADAQEBAAAAAAAAAAAAAAABAgMEBQb/xAAiEQEAAgIBAwUBAAAAAAAAAAAAAQIDESEEEmEFEyIx8EH/2gAMAwEAAhEDEQA/ALxggggCIrjXHVIwjLFU45tZkjxJds+MqFOOcRIw1QXpzKVvqs2w2N61nQDtijqvT3aar3XrxE3iCbG1CHPGblUnydOJ5h6YCwpWs4rr8s3OvTktQpN8ZmWgAp5SeBud3UAY7ClPuav4vqbh50KIHqTFW0nHjkqnZVHOtwffCU3J/W49nZDyMcsrHiTzZvwzawE8FFFtcTVc/vq9kZFFb/OSr95Xsiv140I3Tye/HJWNFX/4wd+AsX3Fb/OKr9qvZB7itfnDWD6V+yK5+zRf44O/Gfs0X+NjvwFi+4rP5wVjtX7IDRmR+EFZ7V+yK7+zRf42O/8A/YwcarH32O/AWCqkND8Iqx2q9kcl01sa/ZHWO1XsiAKxqs/fY78cV4yWfvod6An6pJKfwmq4/fV7I4OMqRqnFdVHWtXsivnMXOH76v8AvQjdxa7r/wDo/wA0BZrM5WZdQFPxcXl38VubSFA9G4GFlJ5UvAqx7iYulkyc1pkmWzdtYO49F4pWZxS4oEFSnL8OeNqVX2qi/wC5+ImvCZBzRCv+ZLdKDzdEB61ZdQ82lxtQWhQuFDcRG8VDyc1iewzWkYWq75mJGaGemzd7hQIvlueiLdG6AzBBBAEEEEAQQQQBAYIDAQDEjIrHKFS5F43lKeyqbcTwzbk37VH0RV2OZxc3PzMws3Liyr0cOwWHoi10JvjPEMwf+XJJQOxUU5ikErV1QEGmjqTCImFszxhzo9Ep71JXU6xPuSksqZ8Fa2TW0UV5cxURceKARe2pvAR+CHxWGJ00p6qS7ktMSrSiCWXgVZc2ULynUAm2h11va0LJjAtZYmUMFEqtRecZWpuaQpLK2xdYcN/FsNYCLRm8SaWwPWJoPGXTKupQcrakTSCHzlzWbsdfFIPDm36QgrlIbpqpANOKX4VIszKs3yVLBJA6NIBovGIUCX03Rqpm0RscoxHUIjYMGGxwjMdFtFO+OcSC8bNqKXEqG8GNIIC8aK2azyd7VPjT1HdD8qriAPHAvzXCh1RdNMmRN0+WmRudaSvtEU3ySHPh+ptHUFlGnei08GKzYWph+gSOyAeoIIIAggggCCCCAIIIICEs/H2J1fQgf5VRTmKk2cVFyM/HGKT9GPqmKexYPfFwEAmt5hfSK+JCRXITVOlKhKl8TKG5grAQ6Ba/iqFwRvSd9oQTfGEXGAmKcZTkxSHKauUk0ocaLKlt50+LnzizYOQG99bbrQoZxROodmnBLyqhNzz8482sKKV7VOVaNCDltu49PPDmFEGFqHriMbzbfCsn9vFb8nLOystTpRmWKitplpx5GyVlCSSQsFdwBfNfXXTcWSeqLtSVKF5CEGWlW5ZGU+UlAIBN+OscXbq3RyykHdF67mOUReCkAZeEcHhGQvhGQkucIaXiP64spuYdGWEZTCRDBRwjsHigRE7Z325TbaRuhrWLGF0w8VQhXe94mqatIIIIuuvPkhWE0eoKUQlIlwVE8BrrE6wpialSdHp1PmZxhuayBCmtu2ooJ3ZgFXHpGkVFQZl2VwDVwyopXMNtS4I32Wux9VxDy3yXeAU33S2i1PhJc0SSbgE/zgL5EZjVpWdtKvnAGNoAggggCCCCAIDBBAQpn42xV+zH1TFP4t+EXFwM/GmKj+h/aYp/Fvwi4Cv5vUmOUnKPTkwhiWQp11xWVCEi5UeiO01oox6A5DsKy9Lw8muzCEqnZy6kKI+Db3C3XYmArSU5IcWzDAd8EQ1pfI4sBXZDRPYQqdLmvBqihUs9a4Q4g+MOcHiOqPUCaql9ynllPvU0tYusWJSEkgjrt2QnqkhI4roz0q+3uUpKSpPjMuDS/X/pFY1MpvS2nmpmguEavIP7pjZygLsbPI7piUUuXSaqJScsMji23QVWAKb316xDiZSnuMreaQpxrKtTjiVkBopbBsL6kFVwLi9rcY025NK8NBczfDI7phfL0Fdh76jsMTUUemB9hG1LhXMttKs5pZasyT/47ekxu1JSglJtS0JbdbzlCLrB8VCToFDnOt+F7boh0U4hD1UFfnUd0w3zVCd4PI7piVKcEIZhwExEwvPKMfY+8T8OjumENQpMxKJK1AKQN6k8ImKCI5TFsukVRpXx3xiF1Wlky82oIFkKGZPRCGLC0sNqQnCEwtwXSl2Xv3lRduIFoVh9ZSLp2avQMp6Dfq0iocDUpVawVXJFrR5UmlbJv8tKrp9Yt6YTUubxXNJYk35mXmG3yltjK6FE5hbdvHZAehZTSVZH6Cf5R2jCQAkAbhGYAggggCCCCAIIIICFs/GeKv1R9QxT2LfhFxcTPxhirqH1DFPYu+FXAV/NGyyY9Ocl04zWOTmQaZXq0wZVy29KgLesEH0x5jmvKiQ4Cx1P4NnFKlvfpR221l1k2V0jmMExOno+VnmJVmWZqcq8mblE5U7NlSgo5ct0W3giFtJQ5LsTc5OJDG2cLykKPwaQANemwuYg0py2YXdlgt9E607bVGzB167xEsacqor0uZCmqTJSS9HSpQLjo5jwA5xx/nWK8r2yRaNRGtmovprOIpt1Ky21MzLzwIFyElRXp02hailsOSTcyiYd2UxsyyklAIChcXBOv7u8a9UQYqUu04lxqbQhaTcKC7EekQ5tVZEwsren1LUVZ7qdJObQX67AdkXY9sF9Ukm5SdUwypSko0JUpKtbn5v8t8J7lAvcxuZyWWSozKCTqSVakxxfm5QJ+Hb7YJjhhcz0mErj1zvhM/PydyPCWu9HDw6U/GG+2CTgh63GBx3NpDeJ+U/GG+2E03WGW0WYVtF9WgiAir7gVNJQPko19MNUbuFTiytRuVG5MPchQD4K3UKzMpp1PcF21LGZ2YH0Te9X6xsnp4QFt8jBtTKh/hh/MxZWCA0MLU1WVAOxGthFYcl9TkUT8nT6XJrEtNKLTzk2vM4tIbWoaJsBqOnSLBpc4pjFj1IQWmaezJMqal0tpSlKlLdBtpfclOnRASwKHCNo5lpB4W6tI1KVo1SSocxgO0EcmnQvfvjrAEEEEAQQQQENZ+MMVdX9hincXfCri4WD9scVDo/sMU9i34RcBX81vMSrk35PZvGMyt1ayxT2SA49a9z80c5/3zRFZreY9SclUmzJYApAl0gbZnbLtxUrU3/l6IBvY5IMGMshtyRfeX5xUytJvz2SQPVERxxyWNUmTcqNCCn5dsZnmHEgrQnnB4gc2+JBVsTTrNYly8qXmFSanBtJdRS2vNbRW/VNonlFdenKW05OPS0wtwHMuXHvZB4DU9UZ0yxeZiHZ1HRZMGOt7fVnmFiUQ84EMy4cWdyUouT6Ic5CmuPISpmQW4kpzBSGbi17X3bo3eWKFimbMuMyZObeQ0BzJUpI9VoWjETTkuph2TLbRRkQhvKoIAVceWLc+vOTGrjaGnrbbQpcitKVpKkKLNgoAZiQbbraxyfpzt2kGQcCnvggWdV9Wmsdk11lts7GXczONJQ6VLBF0sFkZQB0316owqutIm3JxiWWZhaVGzrpKErIAJAFjuvre+o5tQYn5dvKlZYSErF0Eo8oXI09IMcNgz5pHdEO1TnpWdbRkZeZdQV5U5gW7KcUs9O9RENsBz2DPmkd0QkmqWw+k7NIac4KSNPTC+O8hKLn56Xk21ZVPOBOb5o4n0C59EQGqmU6XpdPVWqw0h7x1NyUkvdMOJ0Kl/Rp4j5R054aKlUpuqzjk3PvKefc0KjoAOAAG4DgBpC3FFVFTq7ipfSSlx4PKI4JZTonrJ3k8SSYZ+MBbPJJ8eUn9sv+i5E1xE06jGzc54Ol9gJlm7KUQEqUt4ZtN9s17G4iFcknx7SR9Mv+i5FpVJCXZ+bChcpLCx6FH2wE0EZgggOL7VwVo0UPXGJd0LTrvjuYQve8TII0S5r6YBdBGEm4BjMAQQQGAhUuftlirq/sMU/i74RcW+x8b4q/U/sioMXeWuAgE1vMXtyG4ylZukJw9PPIRNy6j4PmNg4g65esG/otzGKIm95jhLB0vJ8HzbS+mXfAexHKFJmckphpIZTK7QhptACV57Xv2QjxLXKdg+iqcs0hZCvB5ZOhcX0DmvvMUTTsRcoTMoG2KlM7O1hnIUbdZBMMc6/WpmZW/POByYVvW8pRUejXhGdLY5n4y3yRmisd8TrycpCbDVTM3NeNnK1uEDylKBue0wurlRYnGAhp1bytolacyLbFIbCSgHrF+aIxeo80v64z9seaX9casCyA674R/bHmY9cYvUTwl/XALYIR/bHml/XGPtjzS/rgFsLaS8ZcVOZBIXLUx9xCh8lRytg9rghm+2O+0v64V0hM47L19qZCRtKS5kybvFdacPqQYCIQDeIIIgW1ySfHtJ/bL/ouRaM2q1WnB9E39aKt5JPj2k/t1/0XIs2dVavzI52kfWEBOoIIIAhHUxaXSr5qx7IWQiqyrSmX5y0j13/0gO0qq6I7wmkh72IUjdAEYO6MxgwEKY+OMVD6P+yKgxbqtUXAwPt3ikfRD6hin8WeUrqgIBNbzfdE7wHh5S2EOoZDsy6CoX+SBxud0QSa8oxbWBZxkS0usvNtoUwps7RvOhWnkrA4Hnjh660xSI3qJnl6npVInLa2tzEbiC4UioEPESrlmCQ5zg2v6dNdIRVaivrkkOzLBQh34J08Da47REtFUpSX2HGZlaG5F51YaUFK2wUkCySem414Q2VWak36UwnwgOzacgzJSpJyhNjtBuJB3GPK9vHj3bHbmPL3/fy5tY8tPjPif3lXdIk0ztUYk3syUrUQrLvFgTpoebmh2lcOMzWyXt1tN7ZSHhcKKUADKQbC91EJ3cRDTKoM5VwG5gS5cdUUuXy5RqdNRrwha/SZhLitpOuF87QspcbUlTmRAWrMb+IbaDyrkb7WMfRVndY2+OvERaYh3aoMstbSVFwFUmmYUS+lNypLarXKLJ1XzncIHcPshDmRUyuzivfTlSlID2zsQdSrjcc6dLQgq8o/IpZQqdcmGlpOQ6hFhbyTcgp3brdQhCX3ikp2rmUnMRnNiee3PFlUiRhph6ouybTriFpmmmUBSwS4FKVnsco8bKm4Ft4hGmlSx8DbWmczzIbs+ACzmcuEjdewNr6njutDUuZfWoKW+8pQIIUpxRItu1J4axrtXciUbVzIg3QnObJPOBwgNpxtpqadaYWpaEKyhagAVW0J6ib26I70aZZlKow5NW8GXmaf/ZrBSq/QAb+iEZ1JJ4xgi4O6AjdWkHaXU5qQfBDku6ps9Njv9Oh9MJImNUlk4gk0iWSVVqnsgOIGqpqXSLBQ51oFgRvKbHgYhw39EQLZ5JPj6k/t1/0XIsqd1xDMD6NH1hFa8knx5Sv2y/6LkTWs1qXkcc+APXC5htkN2F7lSz/6QFpQQQQAYaqq5nmGmE8PGV/pC6bmW5ZlTjh6hxJ5oa5Btx50vu6rWbmAdZZOVAEdo1SLCNoAgMEYMBDWE/dJiZsbywk260GKbxUrU9UXJPOJp3KC2HtGanKZQTuKkbx2H1RU2OZFyUnpmXUNWllPo4eq0BW8z5RMOWGsQqpSi08CthRvb5sNs4CFqhEd8UvSuSvbb6aYct8N4vSdTC0msRUpxGbwkAHhb2QhqNeafbLcs4EoVoVFQBPVFd6QRyY/T8NLdz0c3rHVZadkzpMWJwS7qXWX0JWncbg+o7/TCg1uZKHEGeGV0WUAECwyhNk6eKMoAsmwsLbogkEdzyk3m6o5OZPCZpLmS9vJTv3nS1ybDU6wn27XnEd4REIICX7drziO8INu15xHeERCCAl+2a86gfvCEs3VJeXQQhQcc4BO70xGoIBS1PTLE6icl3ltTCF50OINik9ESAvUjEQC51xuk1dR8d/KfBplXOoJB2ajxIGW+thEWhRJsqfmW20gm5EBcfJlSJ+n1ylqmpYhsurKXUKDjahsl6haSUntiUrojNc5SXlOPuMOyLUs+hbdsx1dBSb8CCRDFSpSYoXJ09MSzrkvOT7uzlSgkG5shKrfrFR6osiTkpxllsqecU/s0hxw6qUbcTASO9tTCGbqjEv4qTtXPmI17TCBUlMO22zzqhzFRMKZamIR8mASobfnnttM8PJQNyYeJdrZoGkbNMpbGkdYAggggCCCCAjuN6E7WqUkyS9nUJRYflV/pjgegi4iBTrbOMpNQUjwauy42cxLOeKVEf70PERbxiMYowdKVp4Tss6qRqiB4s0yNSOZQ4iA89YgwxOSj7iFsOIUDqlSbGIzMUyZQo+9q7I9FLbxVT0GXq1GYrEunQPS5FyP1VbvQYTqco7ljN4OqDa+OWTWf5QHnNUq+D8ErsjHg73ml92PRf3L/Kw1UAf8G77Ix9yg/B6f/g3vZAedPB3vNOd0wbB7zS+6Y9FXwn+QZ/8AhHvZGb4T/IVQ/hHvZAedNg75pfdMGwd80vumPRf3Jj/oNQ/hXvZB9yf5BqH8K97IDzpsHvNOd0xnwd/zLndMeirYT/INR/hXvZGB9if5v1D+Fe9kB528He8y53TGyZOZV5Mu6f3DHogfYtww7UT/ANm97I3ScNA+Lhmok83gb3sgPPsvRqg+fFlnAniSLARZOAuT5xWWo1UmWkkeMt5emYcyL7+vd2RYTD6AR7i4MmlODyVPsbMJ6buQtawtV8QPJdxVMttSY19z5RRIV+uvS46AIBNQmTi6vMTyGdlQKSQJMcH3Bpcfop4HifRFhZRe5EaS0u1KsoZl20ttNpyoQkWCRzCOsBjKOaACMwQBBBBAEEEEB//Z

i.e. hit the FIRST rather than the THIRD control.

An update - the landline rang again today, 12 hours later, to tell my beloved that my location had changed - I wonder how much longer it's going to do that ……

IBM BPM and Elasticsearch - with added TLS

$
0
0
Following this: -



I've been tinkering further with Elasticsearch on Docker, establishing a TLS connection between it and IBM BPM.

Here's my notes: -

Pull Image


Start container

es=`docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node"docker.elastic.co/elasticsearch/elasticsearch:5.6.3`

Check logs

docker logs $es -f

Upload YAML for Certgen

docker cp ~/instances.yml $es:/usr/share/elasticsearch/config

Generate Self-Signed Certificate, plus Keys

docker exec -i -t $es /bin/bash -c "/usr/share/elasticsearch/bin/x-pack/certgen -in /usr/share/elasticsearch/config/instances.yml -out /usr/share/elasticsearch/certificate-bundle.zip"

Download Certificates

docker cp $es:/usr/share/elasticsearch/certificate-bundle.zip ~

Stop Container

docker stop $es

Remove Container

docker rm $es

Extract and place certificates and key

tar xvzf ~/certificate-bundle.zip --strip-components=1 -C ~/Desktop/elasticsearch-config/x-pack ca/ca.crt

tar xvzf ~/certificate-bundle.zip --strip-components=1 -C ~/Desktop/elasticsearch-config/x-pack node1/node1.crt

tar xvzf ~/certificate-bundle.zip --strip-components=1 -C ~/Desktop/elasticsearch-config/x-pack node1/node1.key

Re-start container

Note; we're mapping ~/Desktop/elasticsearch-config as the ES config root

es=`docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -v /Users/davidhay/Desktop/elasticsearch-config:/usr/share/elasticsearch/config docker.elastic.co/elasticsearch/elasticsearch:5.6.3`

Check logs

docker logs $es -f

Test using Curl - on host

curl --insecure https://localhost:9200 -u elastic:changeme

Should return: -

{
  "name" : "-2S40f4",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "zV8P1a4FR26Q_J_h1E0QKA",
  "version" : {
    "number" : "5.6.3",
    "build_hash" : "1a2f265",
    "build_date" : "2017-10-06T20:33:39.012Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.1"
  },
  "tagline" : "You Know, for Search"
}

or similar

Test using browser

Default credentials are elastic/changeme


Should return same JSON

Test on BPM box

Hostname node1.uk.ibm.com aliased to IP address of host Mac

curl --insecure https://node1.uk.ibm.com:9200 -u elastic:changeme

{
  "name" : "-2S40f4",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "zV8P1a4FR26Q_J_h1E0QKA",
  "version" : {
    "number" : "5.6.3",
    "build_hash" : "1a2f265",
    "build_date" : "2017-10-06T20:33:39.012Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.1"
  },
  "tagline" : "You Know, for Search"
}

or similar

Place CA certificate on BPM box

scp ~/Desktop/elasticsearch-config/x-pack/ca.crt wasadmin@bpm86:~

Update BPM Event Emitter YAML files

vi /opt/ibm/WebSphereProfiles/Dmgr01/config/cells/PCCell1/nodes/Node1/servers/SupClusterMember1/analytics/config/BPMEventEmitter.yml

vi /opt/ibm/WebSphereProfiles/Dmgr01/config/cells/PCCell1/clusters/SupCluster/analytics/config/BPMEventEmitter.yml

ES configuration as follows: -

...
esConfiguration:
    enabled: true
    # The Elasticsearch index name
    index: bpm-events
    # Enable the following properties when Elasticsearch security is on.
    username: elastic
    password: changeme
    httpsTrustType: CRT
    trustFileLocation: /home/wasadmin/ca.crt
    hostnameVerifier: false
    esTaskIndex: restore_task_index
...

Synchronise Node

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython -f fullSync.jy

Validate Sync

ls -al `find /opt/ibm/WebSphereProfiles -name BPMEventEmitter.yml`

-rw-r--r-- 1 wasadmin wasadmins 2793 Oct 19 16:54 /opt/ibm/WebSphereProfiles/AppSrv01/config/cells/PCCell1/clusters/SupCluster/analytics/config/BPMEventEmitter.yml
-rw-r--r-- 1 wasadmin wasadmins 2793 Oct 19 16:54 /opt/ibm/WebSphereProfiles/AppSrv01/config/cells/PCCell1/nodes/Node1/servers/SupClusterMember1/analytics/config/BPMEventEmitter.yml
-rw-r--r-- 1 wasadmin wasadmins 2762 Sep 18 08:51 /opt/ibm/WebSphereProfiles/AppSrv01/installedApps/PCCell1/BPMEventEmitter_war_De1.ear/BPMEventEmitter.war/WEB-INF/classes/BPMEventEmitter.yml
-rw-r--r-- 1 wasadmin wasadmins 2797 Oct 19 17:19 /opt/ibm/WebSphereProfiles/Dmgr01/config/cells/PCCell1/clusters/SupCluster/analytics/config/BPMEventEmitter.yml
-rw-r--r-- 1 wasadmin wasadmins 2797 Oct 19 17:19 /opt/ibm/WebSphereProfiles/Dmgr01/config/cells/PCCell1/nodes/Node1/servers/SupClusterMember1/analytics/config/BPMEventEmitter.yml

All but BPMEventEmitter_war_De1.ear version of file should be the same size/date/time

Start App

/opt/ibm/WebSphereProfiles/Dmgr01/bin/wsadmin.sh -lang jython

AdminControl.invoke('WebSphere:name=ApplicationManager,process=SupClusterMember1,platform=proxy,node=Node1,version=8.5.5.12,type=ApplicationManager,mbeanIdentifier=ApplicationManager,cell=PCCell1,spec=1.0', 'startApplication', '[BPMEventEmitter_war_De1]')

quit

Check Logs

tail -f /opt/ibm/WebSphereProfiles/AppSrv01/logs/SupClusterMember1/SystemOut.log

Note

If you see this: -

Caused by: javax.net.ssl.SSLPeerUnverifiedException: Host name '9.174.27.153' does not match the certificate subject provided by the peer (CN=node1, DC=uk, DC=ibm, DC=com)

use: -

hostnameVerifier: false

in BPMEventEmitter.yml

Backup





Zipping and Tarring on macOS - with added funkiness

$
0
0
So I had a specific requirement yesterday - I wanted to extract three specific files from a ZIP file.

This is what I had: -

unzip -l certificate-bundle.zip

Archive:  certificate-bundle.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  10-19-2017 16:58   ca/
     1310  10-19-2017 16:58   ca/ca.crt
     1679  10-19-2017 16:58   ca/ca.key
        0  10-19-2017 16:58   node1/
     1379  10-19-2017 16:58   node1/node1.crt
     1679  10-19-2017 16:58   node1/node1.key

---------                     -------
     6047                     6 files


So I wanted to extract the certificates and one of the keys …. and place them into specific locations

BUT…..

I didn't want the paths, just the files.

Whilst zip supports this: -

-j  junk paths (do not make directories) 

alas, unzip does not.

Thankfully, the internet had the answer: -

How do I exclude absolute paths for Tar?

I knew that I could use tar on a ZIP file, but this was a nuance.

So here're the commands that I used: -

tar xvzf ~/certificate-bundle.zip --strip-components=1 -C ~/Desktop/elasticsearch-config/x-pack ca/ca.crt
tar xvzf ~/certificate-bundle.zip --strip-components=1 -C ~/Desktop/elasticsearch-config/x-pack node1/node1.crt
tar xvzf ~/certificate-bundle.zip --strip-components=1 -C ~/Desktop/elasticsearch-config/x-pack node1/node1.key


so we use —strip-components to remove the path and -C to place the files into specific locations.

So that's all good then :-)



More on Elasticsearch, Logstash and Kibana (ELK)

$
0
0
Following earlier posts: -




I've had a brief play with a new ( to me ) Docker image, ELK: -


Collect, search and visualise log data with Elasticsearch, Logstash, and Kibana.

using this documentation: -


This time around, I built it using Docker Compose ( on my Mac ) : -

Create a DC YAML

vi docker-compose.yml 

elk:
  image: sebp/elk
  ports:
    - "5601:5601"
    - "9200:9200"
    - "5044:5044"


Spin up the Container

docker-compose up elk

Creating elk_elk_1 ... 
Creating elk_elk_1 ... done
Attaching to elk_elk_1
elk_1  |  * Starting periodic command scheduler cron
elk_1  |    ...done.
elk_1  |  * Starting Elasticsearch Server
elk_1  |    ...done.
elk_1  | waiting for Elasticsearch to be up (1/30)
elk_1  | waiting for Elasticsearch to be up (2/30)
elk_1  | waiting for Elasticsearch to be up (3/30)
elk_1  | waiting for Elasticsearch to be up (4/30)
elk_1  | waiting for Elasticsearch to be up (5/30)
elk_1  | waiting for Elasticsearch to be up (6/30)
elk_1  | waiting for Elasticsearch to be up (7/30)
elk_1  | Waiting for Elasticsearch cluster to respond (1/30)
elk_1  | logstash started.
elk_1  |  * Starting Kibana5
elk_1  |    ...done.
elk_1  | ==> /var/log/elasticsearch/elasticsearch.log <==
elk_1  | [2017-10-20T09:58:07,375][INFO ][o.e.p.PluginsService     ] [Q6xLn7b] no plugins loaded
elk_1  | [2017-10-20T09:58:09,062][INFO ][o.e.d.DiscoveryModule    ] [Q6xLn7b] using discovery type [zen]
elk_1  | [2017-10-20T09:58:09,753][INFO ][o.e.n.Node               ] initialized
elk_1  | [2017-10-20T09:58:09,753][INFO ][o.e.n.Node               ] [Q6xLn7b] starting ...
elk_1  | [2017-10-20T09:58:09,960][INFO ][o.e.t.TransportService   ] [Q6xLn7b] publish_address {172.17.0.2:9300}, bound_addresses {0.0.0.0:9300}
elk_1  | [2017-10-20T09:58:09,974][INFO ][o.e.b.BootstrapChecks    ] [Q6xLn7b] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
elk_1  | [2017-10-20T09:58:13,044][INFO ][o.e.c.s.ClusterService   ] [Q6xLn7b] new_master {Q6xLn7b}{Q6xLn7bNR66inZlv5JcUaQ}{HPqd_E_QSJ2eHModlSUT6A}{172.17.0.2}{172.17.0.2:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
elk_1  | [2017-10-20T09:58:13,080][INFO ][o.e.h.n.Netty4HttpServerTransport] [Q6xLn7b] publish_address {172.17.0.2:9200}, bound_addresses {0.0.0.0:9200}
elk_1  | [2017-10-20T09:58:13,080][INFO ][o.e.n.Node               ] [Q6xLn7b] started
elk_1  | [2017-10-20T09:58:13,143][INFO ][o.e.g.GatewayService     ] [Q6xLn7b] recovered [0] indices into cluster_state
elk_1  | 
elk_1  | ==> /var/log/logstash/logstash-plain.log <==
elk_1  | 
elk_1  | ==> /var/log/kibana/kibana5.log <==


See what's running

docker ps -a

CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS                      PORTS                                                                              NAMES
be3d5ee65642        sebp/elk                   "/usr/local/bin/st..."   2 minutes ago       Up 2 minutes                0.0.0.0:5044->5044/tcp, 0.0.0.0:5601->5601/tcp, 0.0.0.0:9200->9200/tcp, 9300/tcp   elk_elk_1
4f54bc00b67d        websphere-liberty:wlp101   "/opt/ibm/docker/d..."   8 days ago          Exited (143) 45 hours ago                                                                                      dazzling_mestorf


Start a shell on the container

docker exec -it be3d5ee65642 /bin/bash

Pull the logs to the foreground

Note the subtle use of the apostrophe ( ' )

/opt/logstash/bin/logstash --path.data /tmp/logstash/data -e 'input { stdin { } } output { elasticsearch { hosts => ["localhost"] } }'

Sending Logstash's logs to /opt/logstash/logs which is now configured via log4j2.properties
[2017-10-20T10:24:30,729][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/opt/logstash/modules/fb_apache/configuration"}
[2017-10-20T10:24:30,741][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/opt/logstash/modules/netflow/configuration"}
[2017-10-20T10:24:31,388][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2017-10-20T10:24:31,390][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://localhost:9200/, :path=>"/"}
[2017-10-20T10:24:31,516][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2017-10-20T10:24:31,607][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2017-10-20T10:24:31,614][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>50001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "norms"=>false}, "dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "include_in_all"=>false}, "@version"=>{"type"=>"keyword", "include_in_all"=>false}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2017-10-20T10:24:31,620][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost"]}
[2017-10-20T10:24:31,623][INFO ][logstash.pipeline        ] Starting pipeline {"id"=>"main", "pipeline.workers"=>4, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>5, "pipeline.max_inflight"=>500}
[2017-10-20T10:24:31,792][INFO ][logstash.pipeline        ] Pipeline main started
The stdin plugin is now waiting for input:
[2017-10-20T10:24:31,976][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9601}


Send a test message

The Quick Brown Fox Jumped Over The Lazy Dog!

Check the log


{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 6,
    "successful" : 6,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 8,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : ".kibana",
        "_type" : "config",
        "_id" : "5.6.3",
        "_score" : 1.0,
        "_source" : {
          "buildNum" : 15554
        }
      },
      {
        "_index" : "logstash-2017.10.20",
        "_type" : "logs",
        "_id" : "AV85PK0Ji95TIyQOdvFj",
        "_score" : 1.0,
        "_source" : {
          "@version" : "1",
          "host" : "be3d5ee65642",
          "@timestamp" : "2017-10-20T10:03:18.652Z",
          "message" : "this is a dummy entry"
        }
      },
      {
        "_index" : "logstash-2017.10.20",
        "_type" : "logs",
        "_id" : "AV85PT5gi95TIyQOdvFm",
        "_score" : 1.0,
        "_source" : {
          "@version" : "1",
          "host" : "be3d5ee65642",
          "@timestamp" : "2017-10-20T10:03:55.857Z",
          "message" : "I love it !"
        }
      },
      {
        "_index" : "logstash-2017.10.20",
        "_type" : "logs",
        "_id" : "AV85UBqvi95TIyQOdvFp",
        "_score" : 1.0,
        "_source" : {
          "@version" : "1",
          "host" : "be3d5ee65642",
          "@timestamp" : "2017-10-20T10:24:31.867Z",
          "message" : "Hello Fluffy"
        }
      },
      {
        "_index" : "logstash-2017.10.20",
        "_type" : "logs",
        "_id" : "AV85UWpEi95TIyQOdvFr",
        "_score" : 1.0,
        "_source" : {
          "@version" : "1",
          "host" : "be3d5ee65642",
          "@timestamp" : "2017-10-20T10:25:57.808Z",
          "message" : "The Quick Brown Fox Jumped Over The Lazy Dog!"
        }
      },
      {
        "_index" : "logstash-2017.10.20",
        "_type" : "logs",
        "_id" : "AV85PKzri95TIyQOdvFi",
        "_score" : 1.0,
        "_source" : {
          "@version" : "1",
          "host" : "be3d5ee65642",
          "@timestamp" : "2017-10-20T10:03:17.729Z",
          "message" : "this is a dummy entry"
        }
      },
      {
        "_index" : "logstash-2017.10.20",
        "_type" : "logs",
        "_id" : "AV85PK9Si95TIyQOdvFk",
        "_score" : 1.0,
        "_source" : {
          "@version" : "1",
          "host" : "be3d5ee65642",
          "@timestamp" : "2017-10-20T10:03:19.238Z",
          "message" : "this is a dummy entry"
        }
      },
      {
        "_index" : "logstash-2017.10.20",
        "_type" : "logs",
        "_id" : "AV85UCuXi95TIyQOdvFq",
        "_score" : 1.0,
        "_source" : {
          "@version" : "1",
          "host" : "be3d5ee65642",
          "@timestamp" : "2017-10-20T10:24:36.234Z",
          "message" : "Hello Fluffy"
        }
      }
    ]
  }
}

So we have Kibana running: -


and Elasticsearch: -


Next job is to wire my BPM Event Emitter up to this - but that's the easy part :-)

IBM API Connect and the SshClientException

$
0
0
This post represents a frustrating, but extremely enjoyable and interesting, voyage of discovery, digging into problems with SSH on Ubuntu Linux …..

I saw this whilst attempting to create a new Developer Portal for an IBM API Connect 5.0.7.2 implementation: -

For the record, here's the text of the exception: -

Error

Error while performing action add during communication to the Advanced Portal. Please report this error to your server administrator. Error details: com.ibm.apimgmt.api.util.SshClient$SshClientException: An exception occurred during SSH call: com.jcraft.jsch.JSchException: Algorithm negotiation fail.
Error ID: 59ef3f69e4b07d97f084e2ee


I checked the authorized_keys file on the Portal box: -

cat /home/admin/.ssh/authorized_keys 

command="/home/admin/bin/site_action" ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyA1P0bv68VRylLHGwNF+aRYR5FCYAtTJQYRPfbAaE286gPseddNEME0vCxQkAGwqUJX7hZRKNdplw9/o67BHbEedkF6d4O8JPON2ZkFPQTv+cxAhFkDLt86ClRlstvxroqbHYwsRKOl/cOVW/88LEQ90UuQVunUQYanI4A6AJZZ8OvxN+/pgq/bHZULfKF5148IKwY9/90uuavwp6t4Jjm62d2UOHplRv6LiT+qPY2Iykncmqr85X0riUExqwkyyOoVextC450Ui10bMFeQYO4KS0cTHTKd0LuiLUopy4hYmDbyJXNa9t6H6mQVe+P+MjAmJNKx8j4xZqZvojiwUf apim_advanced_portal_ssh_key

which matched that specified within the API Manager Cloud Manager UI: -


I even tried upgrading from the older version of the Developer Portal ( 5.0.7.2 based upon Debian 7 ) to the latest fix pack ( 5.0.8.0 based upon Ubuntu 16.0.4.3 LTS ), but to no avail.

As this is a test environment, running on my own Beast box, I set the Portal to trust ALL certificates: -

set_apim_cert -i

WARNING: This should only be used for development and testing purposes as it is not secure and leaves the Developer Portal exposed to a man-in-the-middle attack.

and checked the status: -

status

Operating System: Ubuntu 16.04.3 LTS
System version: 7.x-5.0.8.0-20170908-0855
Distribution version: 7.x-5.0.8.0-20170907-2206

Free disk space: 22G
 RAM Free/Total: 1941 MB / 3951 MB (49% free)
   Set Hostname: OK
     DNS Server: Reachable (8.8.8.8)
   APIC SSH Key: OK

Configuration:
  APIC Hostname: management.uk.ibm.com
  APIC IP: 192.168.1.150
  Devportal Hostname: portal.uk.ibm.com
  Devportal IP: 192.168.1.151
  APIC Certificate Status (Insecure): WARNING - Only suitable for development and PoC purposes.

Node is standalone

Site web check: All sites OK

Site services:
         Webhooks: All sites Up
  Background sync: All sites Up

Services:
  Queue                      is Up
  Database   [Mysql]         is Up (Standalone)
  Web Server [Nginx]         is Up
  PHP Pool   [Php7.0-fpm]    is Up
  Inetd      [Openbsd-inetd] is Up
  REST       [Restservice]   is Up

SUCCESS: All services are Up.


It took me a while, but I worked out how to debug …

Having switched to the Ubuntu version of the Developer Portal, I was able to turn on debugging in the SSH Daemon ( SSHD ), by editing the sshd_config file: -

sudo vi /etc/ssh/sshd_config 

and changing the logging level from: -

# Logging
SyslogFacility AUTH
LogLevel INFO

to: -

# Logging
SyslogFacility AUTH
#LogLevel INFO
LogLevel DEBUG3


I then restarted the SSHD service: -

sudo /etc/init.d/ssh restart

[ ok ] Restarting ssh (via systemctl): ssh.service.

and watched the logs whilst I reproduced the problem: -

tail -f /var/log/auth.log 

which gave me: -

...
Oct 25 08:55:12 portal sshd[24777]: debug1: kex: algorithm: (no match) [preauth]
Oct 25 08:55:12 portal sshd[24777]: fatal: Unable to negotiate with 192.168.1.150 port 52443: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1,diffie-hellman-group-exchange-sha1 [preauth]
Oct 25 08:55:12 portal sshd[24777]: debug1: do_cleanup [preauth]

...

I then checked the sshd_config file again: -

sudo vi /etc/ssh/sshd_config 

and looked at the KexAlgorithms line: -

KexAlgorithms ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1

which tied up nicely …. which was confusing :-(

And then …. 

IT JUST STARTED WORKING !!!

To prove it, I deleted the Developer Portal VM, and built it again from the original .OVA file ….

AND IT WORKED !!

First time, out of the box …

So I'm at a complete and utter loss to know what broke …

However, I learned shedloads in diagnosing the problem, so that's all good then :-)

For the record, the Ubuntu version of the Developer Portal ( 5.0.8.0 ) uses SSH-2.0-OpenSSH_7.2p2 - in case it becomes relevant down the line ….

I'm now going to tinker some more, before ditching the APIC Management Server, and rebuilding that with the new 5.0.8.0 OVA file …

In the context of 5.0.8.0, this is what I've downloaded …

APIConnect_Management_5.0.8.0_20170905-1133_a7fe4cd1d442_c04798a.ova (2.99 GB)

5.0.8.0-APIConnect-Portal-Ubuntu16-20170908-0855.ova (879.64 MB)

from IBM Fix Central, as per this: -

Please note: The Linux distribution for the Developer Portal OVA has moved from a Debian V7 base to an Ubuntu V16.04 base. Support for the Debian V7 OVA is being withdrawn in May 2018. You are strongly encouraged to migrate your Developer Portal to the Ubuntu V16.04 base now, as support for Debian V7 upgrades will be removed by May 2018.


IBM API Connect - Debugging

$
0
0
This is another of those work-in-progress posts, but I'm hitting an issue testing an API that I've created using IBM API Connect 5.0.7.2.

I've developed the API using an existing Web Service running on IBM Bluemix ( it's actually the IBM ODM Rules Service ), and this is a long-used Hello World Rule that I created a few years ago ( I even have a post or two for that ).

I


During the debugging phase, I wanted to check the logs that the API Manager ( aka Cloud Manager Console - CMC ) was producing.

I'm logged into the CMC via SSH: -

ssh admin@management

and am watching the CMC logs go by: -

debug tail file /var/log/cmc.out

...
2017-10-26 13:43:02.666 SEVERE [T-2226] [com.ibm.apimgmt.exception.APIGenericException$Serializer.serialize] Error(59f1e666e4b07d97f084e8ff), Http-Code(401), Message(The HTTP request requires user authentication. Please repeat the request with a suitable Authorization header field.), User(), Path(get:/catalogs/59f1e293e4b07d97f084e7fd/webhooks)

I realised that, during the API Assembly phase, I'd NOT specified the credentials that the ODM Rule Service requires.

I fixed this by selecting the Proxy component in my assembly diagram: -


and entering the credentials: -


This has got me further forward ….

… it's still not working, but I suspect that's a problem between the API Manager and the DataPower Gateway ….


IBM API Connect and IBM DataPower Gateway - The Fun Continues

$
0
0
As per previous posts, I'm continuing to enjoy the voyage of discovery that is IBM API Connect (APIC) and IBM DataPower Gateway (IDG).

This time it's me trying to understand (a) why things aren't properly working and (b) how APIC drives IDG

Thus far, I've discovered that API connects to the XML Management Interface on the IDG, and creates a new Domain ( with a semi-random name prefixed by APIMgmt_e.g. APIMgmt_BF8B3A8C34 ), and then creates a pair of Multiprotocol Gateways (MPG), named webapi-internal and webapi.

All seems fine but ….

I'm seeing this in the IDG logs: -

1,20171026T152710.907Z,APIMgmt_BF8B3A8C34,network,error,xmlmgr,webapi-wcc,36511,,,0xb30009,,,"Host connection could not be established"
1,20171026T152710.907Z,default,network,error,,,1759,,,0x80e00173,,,"TCP connection attempt refused from 127.0.0.1 to 127.58.140.52 port 2444"
1,20171026T152710.907Z,APIMgmt_BF8B3A8C34,network,error,xmlmgr,webapi-wcc,36511,,,0x80e00049,,,"Host connection failed to establish: 127.58.140.52 : tcp port 2444"
1,20171026T152710.907Z,APIMgmt_BF8B3A8C34,network,error,,,36511,,,0x80e00627,,,"Error occurred (port error) when connecting to URL 'http://127.58.140.52:2444/ODCInfo/ODCInfo?c=analytics-lb'"
1,20171026T152710.907Z,APIMgmt_BF8B3A8C34,wcc,warn,wcc-service,webapi-wcc,36511,,,0x80e0053c,,,"Request for WebSphere Cell information failed: Empty result set"


which is weird because my IDG has a static IP address of 192.168.1.200.

( How I got that address is a whole other blog post - ask me about VMware, Bridged networking and DHCP ! )

So I dug further into the configuration, via the IDG UI: -


and then searched for that particular IP address - 127.58.140.52 - which occurred in three places: -





So I dug into the configuration further: -


and found this: -


I changed the address to the correct address of the IDG ( yes, and I've NO idea why APIC / IDG things that we have a WebSphere Application Server (WAS) cell in the mix )….

I'm not yet fully there … but I'm learning as I go.

For the record, I did some other tinkering to IDG: -

Add hostname aliases for the APIC Management and Developer Portal boxes

config; dns; static-host management.uk.ibm.com 192.168.1.150; static-host portal.uk.ibm.com 192.168.1.151; static-host datapower.uk.ibm.com 192.168.1.200; exit; write mem; exit;

config; dns; show; exit; exit

Global configuration mode
Modify DNS Settings configuration

 admin-state enabled 
 name-server 8.8.8.8 53 53 3 
 static-host datapower.uk.ibm.com 192.168.1.200 "" 
 static-host localhost 127.0.0.1 "" 
 static-host management.uk.ibm.com 192.168.1.150 "" 
 static-host portal.uk.ibm.com 192.168.1.151 "" 
 force-ip-preference off 
 load-balance round-robin 
 retries 2 
 timeout 5 Seconds


test tcp-connection management.uk.ibm.com 443

TCP connection successful

test tcp-connection portal.uk.ibm.com 443

TCP connection successful

test tcp-connection datapower.uk.ibm.com 8443

TCP connection successful

Check the internal IDG load balancers

config; show domains

 Domain             Needs save File capture Debug log Probe enabled Diagnostics Command Quiesce state Interface state Failsafe mode 
 ------------------ ---------- ------------ --------- ------------- ----------- ------- ------------- --------------- ------------- 
 APIMgmt_BF8B3A8C34 off        off          off       off           off                               ok              none          
 default            off        off          off       off           off                               ok              none          


switch domain APIMgmt_BF8B3A8C34

show loadbalancer-st

 Group        Host          Port Operational state Weight Administrative state 
 ------------ ------------- ---- ----------------- ------ -------------------- 
 analytics-lb 192.168.1.150 9443 up                20     enabled              
 mgmt-lb      192.168.1.150 0    up                20     enabled              




IBM Cloud Private - Docker, Ubuntu and Volumes

$
0
0
So this week I'm tinkering ( I love that word ) with IBM Cloud Private (ICP), and am planning to install the Community Edition (CE) variant on an Ubuntu VM on my Mac.

This is what I have: -
  • macOS 10.13 High Sierra
  • VMware Fusion 10.0.1
  • Ubuntu 17.10
  • Docker 17.0.6.1-ce
  • IBM Cloud Private 2.1.0
and I'm following the ICP installation from here: -


Having pulled the image: -

sudo docker pull ibmcom/icp-inception:2.1.0

Having previously created a target installation directory: -

sudo mkdir /opt/ibm-cloud-private-ce-2.1.0

and changed to that directory: -

cd /opt/ibm-cloud-private-ce-2.1.0

I then tried to start the image: -

sudo docker run -e LICENSE=accept \
  -v "$(pwd)":/data ibmcom/icp-inception:2.1.0 cp -r cluster /data

However, this didn't appear to do anything :-( 

I then dug further in: -

sudo bash
cd /opt/ibm-cloud-private-ce-2.1.0
docker run -e LICENSE=accept \
  -v "$(pwd)":/data ibmcom/icp-inception:2.1.0 cp -r cluster /data

which returned: -

docker: Error response from daemon: error while creating mount source path '/opt/ibm-cloud-private-ce-2.1.0': mkdir /opt/ibm-cloud-private-ce-2.1.0: read-only file system.

This made no sense, given that I'm effectively running as root :-(

I experimented further: -

docker run -it -v /opt/ibm-cloud-private-ce-2.1.0:/data -e LICENSE=accept ibmcom/icp-inception:2.1.0 /bin/bash

which resulted in much the same: -

docker: Error response from daemon: error while creating mount source path '/opt/ibm-cloud-private-ce-2.1.0': mkdir /opt/ibm-cloud-private-ce-2.1.0: read-only file system.

So, for the record, the switch -v /opt/ibm-cloud-private-ce-2.1.0:/data means that the local OS path ( /opt/ibm-cloud-private-ce-2.1.0 ) is being mapped to the local-to-the-container path ( /data ).

I Googled about a bit: -


which made me wonder whether the problem was with the location, rather than the permissions of the target directory.

I tested this theory: -

mkdir ~/ibm-cloud-private-ce-2.1.0
cd ~/ibm-cloud-private-ce-2.1.0
sudo docker run -e LICENSE=accept \
  -v "$(pwd)":/data ibmcom/icp-inception:2.1.0 cp -r cluster /data

This worked without error, and I was able to confirm that the last part of the command: -

cp -r cluster /data

( which copies data OUT of the container INTO the local filesystem, as mapped using the -v switch )

This is how I validated it: -

pwd

/home/dave/ibm-cloud-private-ce-2.1.0

ls ~/ibm-cloud-private-ce-2.1.0/ -R

/home/dave/ibm-cloud-private-ce-2.1.0/:
cluster

/home/dave/ibm-cloud-private-ce-2.1.0/cluster:
config.yaml  hosts  misc  ssh_key

/home/dave/ibm-cloud-private-ce-2.1.0/cluster/misc:
ldap  storage_class

/home/dave/ibm-cloud-private-ce-2.1.0/cluster/misc/ldap:
cacert  keystone.ldap.conf

/home/dave/ibm-cloud-private-ce-2.1.0/cluster/misc/ldap/cacert:

/home/dave/ibm-cloud-private-ce-2.1.0/cluster/misc/storage_class:

So, the moral of the story appears to be that, for Docker on Ubuntu, it's not possible to map volumes from a container to a directory that's NOT in the user's home directory.

I'll dig further …..

Meantime, I can now continue with my ICP implementation ...

For the record, here's how I checked the versions on the Ubuntu VM: -

lsb_release -a

No LSB modules are available.
Distributor ID:Ubuntu
Description:Ubuntu 17.10
Release:17.10
Codename:artful


docker images

ibmcom/icp-inception   2.1.0               fa65473d72d8        7 days ago          445 MB

docker version

Client:
 Version:      1.13.1
 API version:  1.26
 Go version:   go1.8.3
 Git commit:   092cba3
 Built:        Thu Oct 12 22:34:44 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.06.1-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   5ff8f9c
 Built:        Fri Aug 18 14:48:14 2017
 OS/Arch:      linux/amd64
 Experimental: false

IBM WebSphere Application Server - Tuning and Monitoring and Tuning - IBM Health Center

$
0
0
This will be the first of a number of posts, as I'm currently engaged with a few WAS performance gigs, so am refreshing my memory on some of the tooling.

This time around, I'm looking at the Health Centre, which has been part of IBM Java since version 5.


Health Center is a very low overhead monitoring tool. It runs alongside an IBM Java application with a very small impact on the application's performance (less than 1%). Health Center monitors several application areas, using the information to provide recommendations and analysis that help you improve the performance and efficiency of your application. Health Center can save the data obtained from monitoring an application and load it again for analysis at a later date.

The plan is to turn on the Health Centre "agent" within a Java Virtual Machine (JVM) underlying IBM BPM Advanced, specifically the AppCluster member, and then analyse the output using Eclipse.

I'm running WAS 8.5.5.12 and BPM 8.6, on IBM Java 8.

Whilst the HC agent is part of IBM Java, the UI tool is part of the IBM Support Assistant, and can be installed into Eclipse.

I'm using the latest Eclipse Oxygen and I simply searched for the Health Centre tool within the Eclipse Marketplace, by searching for the word "health", which returned one hit: -


Having installed the plugin, and restarted Eclipse, I then added the necessary instrumentation to my BPM JVM, changing the Generic JVM Arguments from: -

${IBMSCMX} ${IBMGCPOLICY_GENCON} ${IBMJITPMC} -Xmns256m -Xmnx768m -Xgc:preferredHeapBase=0x100000000 -Xdisableexplicitgc -Xss2048k -Dsun.net.http.allowRestrictedHeaders=true -Declipse.bundle.setTCCL=false

to: -

${IBMSCMX} ${IBMGCPOLICY_GENCON} ${IBMJITPMC} -Xmns256m -Xmnx768m -Xgc:preferredHeapBase=0x100000000 -Xdisableexplicitgc -Xss2048k -Dsun.net.http.allowRestrictedHeaders=true -Declipse.bundle.setTCCL=false -Xhealthcenter

I then restarted the AppCluster, and monitored the SystemOut.log file as the JVM restarted: -

tail -f /opt/ibm/WebSphereProfiles/AppSrv01/logs/AppClusterMember1/SystemOut.log

[01/11/17 12:47:37:314 GMT] 00000001 WsServerImpl  A   WSVR0001I: Server AppClusterMember1 open for e-business


In Eclipse, I established a new Connection: -






As can be seen, this iterated through a range of ports, starting at 1972 and running to 2072.

Alas, I ended up with this: -


and: -


Following the advice in the "Unable to contact agent" dialogue, I changed the Generic JVM Arguments for the AppCluster member from: -

${IBMSCMX} ${IBMGCPOLICY_GENCON} ${IBMJITPMC} -Xmns256m -Xmnx768m -Xgc:preferredHeapBase=0x100000000 -Xdisableexplicitgc -Xss2048k -Dsun.net.http.allowRestrictedHeaders=true -Declipse.bundle.setTCCL=false -Xhealthcenter

to: -

${IBMSCMX} ${IBMGCPOLICY_GENCON} ${IBMJITPMC} -Xmns256m -Xmnx768m -Xgc:preferredHeapBase=0x100000000 -Xdisableexplicitgc -Xss2048k -Dsun.net.http.allowRestrictedHeaders=true -Declipse.bundle.setTCCL=false -Xhealthcenter:transport=jrmp

For the record, I'm running Eclipse Oxygen on macOS, and am thus using Oracle's Java 8, rather than an IBM JRE.

I restarted the AppCluster ….

… and then re-tried the New Connection process again

This time around, I got this: -


and this: -


Now I have full access to the various HC metrics from the AppCluster JVM e.g.




etc.

Which is nice :-)

For the record, there's some useful insight about the HC Monitoring Agent here: -


Hope this helps.

IBM WebSphere Application Server - Tuning and Monitoring and Tuning - IBM Garbage Collection and Memory Visualizer (GCMV)

$
0
0
Following on from an earlier post: -


I've also been looking at Garbage Collection and Memory Visualizer (GCMV), which is another part of the overall IBM Support Assistant (ISA) tool, and which also plugs nicely into Eclipse.

As per the previous post, I'm using Eclipse Oxygen, and added GCMV via the Help -> Install New Software 


from this location: -


Having restarted Eclipse, I switched to the GCMV perspective, via 


and then chose File -> Load File: -


and brought in a native_stderr.log file that I'd previously exported from my BPM 8.6 / WAS 8.5.5.12 / Java 8 environment: -


Immediately, GCMV gave me a useful report: -




There's also a set of templates: -

which gives me a lot into which to get my teeth ….



WebSphere Application Server Performance Cookbook

$
0
0
Building upon my earlier posts: -



We have this: -


The WebSphere Application Server Performance Cookbook covers performance tuning for WebSphere Application Server (WAS), although there is also a very strong focus on Java, Operating Systems, and theory which can be applied to other products and environments.

This is amazingly useful, partly because it's available as a PDF as well an online HTML reference.

Chapters include: -





etc.

Definitely worth adding to the reading list ...
Viewing all 1851 articles
Browse latest View live


Latest Images

<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>