Long story. I decided to upgrade my server OS. Everything, almost everything, went fine. But, when I try to access my website, I couldn't. I just update some nginx settings and reload the configuration file, and it worked. Nothing fancy, quite easy, actually.

I visited my web site and tried to reach my blog, but I couldn't. I thought, I just forgot to restart forever. Then, I did but nothing happened. I started Ghost manually and saw the problem. Node were upgraded and my old Ghost blog didn't support this new version.

I should check Ghost/nodeJS version compatibility but I didn't. Actually, I should have a Dockerised environment of my droplet. By doing so, I would have known what could happen if I upgrade my server operating system. A zero-cost environment could saved me a lot of time and stress. Huge mistake.

Crying over the spilled milk never is a good choice. My choices were, either downgrade nodeJS and keep my old Ghost version or upgrade Ghost. I chose the latter. I installed the Ghost client, MySQL and the latest Ghost version. Nice and easy, at first, though. I tried to import my old blog post into my new Ghost, but I couldn't. Another issue came out. Ghost asked me for a JSON backup and I haven't done that before. I thought a database backup was enough, but it wasn't. Another mistake to my bag.

Looking for information about Red, Blue and Purple teams, I found out an interesting article written by Daniel Miessler titled The Difference Between Red, Blue, and Purple Teams.

There, the author extraordinarily condensed into a single image all the INFOSEC color meanings.

Originally posted at https://danielmiessler.com/images/BAD-pyramid-miessler.png

As you can see, the author referenced April Wright work at the image footer. Thanks to that, I searched and downloaded her BlackHat 17 USA presentation Orange is the new purple. There, April introduced the Yellow colour to represent the builders which led to new derived colours: Orange and Green. Just like Daniel, I consider, Orange and Green a mindset rather than a fully dedicated team.

Personally, I am a builder who in 2016 received a Penetration Testing report pointing some security issues affecting my applications. Luckily for me, I could talk to the pentesters openly to interchange ideas and opinions about information security. Those conversations helped me not only to understand the issues and correct them effective and efficiently but also to include the information security from the very beginning in the development cycle.

What I did was to start to think as an attacker trying to get access to my applications as well as a defender building defense mechanisms to deter, detect and monitor the more possibles attacks. In few words, I was a member of the Yellow team whose mindset sometimes has to turn Orange or Green to build more secure and resilient applications.

Back in 2018, I began to pursue an MSc in Digital Investigation & Forensic Computing at University College Dublin. Now, I am sure I could join a Red, Blue or Purple team.

Additional links

• Introducing the InfoSec colour wheel - blending developers with red and blue security teams.
• Architect Security with April Wright.

Splunk is a platform that let us look into huge sets of data generated by several sources.

The official image is available at https://hub.docker.com/r/splunk/splunk/.

This is a ready-to-use docker-compose file to deploy Splunk wherever you want.

version: '3'

services:
  vsplunk:
    image: busybox
    volumes:
      - "/opt/splunk/etc"
      - "/opt/splunk/var"
  
  splunkapp:
    hostname: splunkapp
    image: splunk/splunk:latest
    environment:
      SPLUNK_PASSWORD: ${RANDOMPASS}
      SPLUNK_START_ARGS: --accept-license
      SPLUNK_ENABLE_LISTEN: 9997
      SPLUNK_ADD: tcp 1514
    volumes_from:
      - vsplunk
    ports:
      - "8000:8000"
      - "9997:9997"
      - "8088:8088"
      - "1514:1514"

Feel free to copy or clone it from -> https://github.com/immontilla/splunk

If you decided to copy it, you will need to assign a password into an environment variable named RANDOMPASS.

In a UNIX based system, you can issue this command:

export RANDOMPASS=`date +%s | sha256sum | base64 | head -c 32 ; echo`

Do not forget to review its value because you will need it when you want to access http://localhost:8000/ as the admin user.

I wanted to watch off-line some Adobe Connect recordings in my commuting time. Googling about it, I found out that you can download any Adobe Connect recording just by appending output/filename.zip?download=zip to its url.

The resulting zip file is just a mix of FLV (video) and XML (configuration) files. In Windows, FLV files could be played using VLC media player.

This is just a simple Powershell script to automate this task

You can run it this way:

.\adbconnect2zip.ps1 -filename {desired file name} -url {recording url link}

don't forget to replace {desired file name} and {recording url link} appropriately before running.

Alternatively, you can use curl or wget to do the same thing in Linux, Mac or Windows:

wget -O {desired file name} {recording url link}

curl -o {desired file name} {recording url link}

curl and wget are common Unix utilities. In Mac OS and Windows, both utilities could be installed using package managers like brew and chocolatey, respectively.

Update

Using ffmpeg we can extract MP3 audio from FLV files by issuing this:

ffmpeg -i {flv filename} -acodec libmp3lame -b:a 192k {mp3 filename}

replacing {flv filename} and {mp3 filename} appropriately.

FFmpeg Windows 10 builds could be downloaded from https://ffmpeg.zeranoe.com/builds/.

Using 7-Zip we can extract only the FLV files from the recording zip file downloaded before by running:

7z e {zip file} -o{extract directory} *.flv -r

replacing {zip file} and {extract directory} appropriately. Note: there is no space between -o and the extract directory name.

I have switched my machine from Linux Mint to Debian Stretch keeping my beloved Cinnamon as desktop environment. The process itself, was not a big deal. However, I did some additional tasks to suit my taste.

Scenario

First of all, I am going to share my scenario:

• My laptop will have only one operating system, Debian.

• My laptop disk will be fully encrypted.

• I will be the only laptop user.

• I will never add my laptop user to the sudo group.

With this scenario, I had to make some changes in the Debian default configuration to have a better user experience

Auto-boot to Debian

I don't like to choose my operating system every time I turn on my laptop because there is only one option, so I did this:

• Log in as root
• Edit the /etc/default/grub file
• Set the GRUB_TIMEOUT to 0

(GRUB_TIMEOUT=0)

• Save the file, and then run:

update-grub

Turn off the annoying system bell sound

I really hate the sound I heard when I press the TAB key. I don't understand why is turned on by default. Well, to turn this s**t off, logged as root I had to:

• Edit the /etc/inputrc file and uncomment the line

set bell-style none

• Edit the ~/.bashrc file and append this:

if [ -n "$DISPLAY" ]; then
  xset b off
fi

• Run

source ~./profile

Enable auto-login

My user can't sudo, and my encryption key is sufficient strong and secure, so I don't want to authenticate every time I turn on my laptop. Well, I manage to auto-login on my laptop by doing this:

• Log in as root
• Open the /usr/share/lightdm/lightdm.conf.d/01_debian.conf file, and
• Append this lines in it, replacing username with yours.

[SeatDefaults]
autologin-user=username
autologin-user-timeout=0

Conclusions

If you don't like something, change it. As simple as it sounds. Is Linux.

References

• How to enable auto-login in Debian 9 Xfce:
  https://steemit.com/software/@kskarthik/how-to-enable-auto-login-in-lightdm
• Debian - How to turn off the system bell
  https://blog.sleeplessbeastie.eu/2012/12/28/debian-how-to-turn-off-the-system-bell/

In this post, I will show you how to code in Go & mongoDB a small API using a couple of Go tool-kits: Goji and mgo. As usual, find the code at Github: https://github.com/immontilla/mgo-n-goji.git.

Before to start, in case you have never give Go a try, I suggest you to check https://tour.golang.org/ first, and back here afterwards. If you want to learn more about mongoDB before to continue as well, you have a great set of free courses at MongoDB University, pick some from its catalogue and return here later.

Welcome back from your learning tour...

API Specification

Our phone book API will meet these requirements:

• A contact should have a unique nick.
• The nick has to be alphanumeric with a length greater or equal than 3 and lower or equal than 36.
• A contact should have at least one mobile number associated.
• A contact could have one or more email addresses associated.

Pre-requisites

• Go
  
  You can find the official installation instructions here. I have installed: https://dl.google.com/go/go1.10.3.linux-amd64.tar.gz on my Linux Mint machine.
  Ubuntu users have an installation wiki at https://github.com/golang/go/wiki/Ubuntu.

• mongoDB
  
  If you want to install it on your machine, follow this guide: https://docs.mongodb.com/manual/administration/install-community/.
  If you are an Ubuntu user you have this specific guide https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/.
  You could create an Atlas Free Tier Cluster following https://docs.mongodb.com/manual/tutorial/atlas-free-tier-setup/.
  Last but no least, you could use this Vagrant file: https://s3.amazonaws.com/edu-downloads.10gen.com/M040_2018_August/static/handouts/m040/environment_setup.zip. This file belongs to the "M040: New Features and Tools in MongoDB 4.0" course and contains a ready-to-use MongoDB 4.0 instance.

Go tool-kits

• Goji: a HTTP request multiplexer which compares the incoming requests to a list of registered Patterns dispatching to the Handler that corresponds to the first matching Pattern.
• mgo: a MongoDB driver for the Go language. I am using this fork: github.com/globalsign/mgo because the original project is unmaintained.

Installation

To clone:

cd $GOPATH/src && git clone https://github.com/immontilla/mgo-n-goji.git

To install:

cd $GOPATH && go install mgo-n-goji/cmd/app/

To test:

cd $GOPATH/src/mgo-n-goji/tests/postman/ && npm install && npm run test-n-report

To see the test results open your browser at http://localhost:4444/

Pending

• Swagger documentation
• Testing - BDD

A new security header has been added to the https://securityheaders.com/ checking service, the Feature Policy header.

The Feature Policy is a new header that allows a site to control which features and APIs can be used in the browser. In my case, on my both websites, I had blocked everything except the fullscreen feature.

This is the line I inserted in my ngnix configuration file to do that:

add_header Feature-Policy "geolocation none;midi none;notifications none;push none;sync-xhr none;microphone none;camera none;magnetometer none;gyroscope none;speaker none;vibrate none;fullscreen self;payment none;";

References

• Feature Policy - Draft Community Group Report, 26 July 2018 https://wicg.github.io/feature-policy/
• Introduction to Feature Policy by Eric Bidelman https://developers.google.com/web/updates/2018/06/feature-policy
• A new security header: Feature Policy <a href="https://scotthelme.co.uk/a-new-security-header-feature-policy/" title="A new security header: Feature Policy" https://scotthelme.co.uk/a-new-security-header-feature-policy/target="_blank">https://scotthelme.co.uk/a-new-security-header-feature-policy/

Analyse your HTTP response headers

Scan your site now at https://securityheaders.com/ and get a decent grade like A or A+, please.

Originally read at "The Business Blockchain", a William Mougayar book...

When the wind of change blows, some people build walls, others build windmills.

I'm starting to delve into blockchain from my developer point of view. Maybe the following warning is not absolutely necessary but just in case: This post is not about cryptocurrencies, it is about the Hyperledger Composer.

According to the Wikipedia, Hyperledger is an umbrella project of open source blockchains and related tools, started in December 2015 by the Linux Foundation to support the collaborative development of blockchain-based distributed ledgers.

One of these open source blockchain related tools is the Hyperledger Composer. If you want to give it a try, your machine have to meet some pre-requisites. If you are an Ubuntu user like me, please follow this link. The installation instructions are here. Its beginner tutorial is here.

My advice: stop reading this post now and follow the beginner tutorial by your own. Afterwards, take a look at my repository https://github.com/immontilla/hyperledger-composer-bank, where you can find a basic/minimal implementation of a bank.

To see it quickly in action, open a shell and run these commands:

git clone https://github.com/immontilla/hyperledger-composer-bank && cd hyperledger-composer-bank && npm install && npm test

You will see something like this:

As you can see, I have added, some testing. Sadly, the tutorial do not include it or even mention it. This is wrong, testing is one important part of any software project.

Finally, let me invite you to play with Hyperledger Composer too. Think about some actions (transactions) someone (participant) like/need/want to do with its stuff (assets) and implement it using the Hyperledger Composer.

I upgraded my Spring Boot projects on Github from the 1.5.x version to the newest 2.0 over the weekend. These are my experiences:

immontilla/file-uploading-web-app

• webjars-locator dependency missing

I had to change the dependency org.webjar.webjars-locator to org.webjar.webjars-locator-core. I found out how to solve it in this article: Spring-Boot-2.0-Migration-Guide#webjars-locator.

• Spring 5 - MockMvcRequestBuilders.fileUpload deprecation

In Spring 5, MockMvcRequestBuilders.fileUpload has been deprecated in favour of MockMvcRequestBuilders.multipart. Read the details here: Deprecated List (Spring Framework 5.0.3.RELEASE API).

I had to change this:

MockMultipartFile csvFile = new MockMultipartFile("file", "test", "text/csv", "a,b,c,d,e,f".getBytes());
this.mvc.perform(fileUpload("/upload").file(csvFile)).andExpect(status().isBadRequest()).andExpect(jsonPath("$.validFileExtension").value("false"));

to:

ResultMatcher badRequest = MockMvcResultMatchers.status().isBadRequest();
MockMultipartFile csvFile = new MockMultipartFile("file", "test", "text/csv", "a,b,c,d,e,f".getBytes());
MockHttpServletRequestBuilder uploadRequest = MockMvcRequestBuilders.multipart("/upload").file(csvFile);
mockMVC.perform(uploadRequest).andExpect(badRequest).andExpect(jsonPath("$.validFileExtension").value("false"));

Honouring "Uncle Bob", I tweaked a little the test code readability as well.

• Upgraded project dependencies

These changes are not related to the Spring Boot version upgrade. Its only goal is to keep the project's dependencies up-to-date.
org.apache.tika.tika-core was upgraded from 1.16 to 1.17

com.google.guava.guava has been split in two versions jre and android. Formerly was just 23, now is 24.0-jre

org.webjars.bootstrap is not an alpha version anymore. I upgraded it from 4.0.0-alpha.6-1 to 4.0.0-2 version

org.webjars.jquery was upgraded from 3.2.1 to 3.3.1-1

org.webjars.tether was upgraded from 1.4.0 to 1.4.3

immontilla/spring-boot-https-server

• Plugin io.spring.dependency-management

I had to add the io.spring.dependency-management plugin to the build.gradle file. Without it, the building process will fail.

• ErrorAttributes and ErrorController classes

In Spring Boot 2.0, the ErrorAttributes and ErrorController classes has been packaged in org.springframework.boot.autoconfigure.web. In Spring Boot 1.5.x, was packaged in org.springframework.boot.web.servlet.error.

• Upgraded project dependencies

org.webjars:webjars-locator:0.32-1 was changed to org.webjars.webjars-locator-core.

org.webjars.bootstrap was upgraded from 4.0.0-alpha.6-1 to 4.0.0-2 version.

com.google.code.gson was upgraded from 2.8.1 to 2.8.2 version.

immontilla/wedeploy-norrisws

This project required the same fixes than immontilla/spring-boot-https-server except any dependency upgrade.

Final thoughts

I recommend to apply the Spring Boot upgrade sooner than later. At least in my case, it was a really straightforward process.

I have found an artistic, fun and original article about HTTP codes published at Medium by Hani Lim. I am going to share its images here.

Even bad code can function


Taken from Clean Code: A Handbook of Agile Software Craftsmanship by Robert C. Martin

There may be cases where you do not want to store a container’s data on the host machine, but you also don’t want to write the data into the container’s writable layer, for performance or security reasons.

_{Use tmpfs mounts https://docs.docker.com/engine/admin/volumes/tmpfs/}

That's why I removed the VOLUME instruction in the immontilla/secure-file-uploader's Dockerfile.

FROM alpine/git as clone
LABEL maintainer="Iván Mauricio Montilla Figueroa"
WORKDIR /app
RUN git clone --progress https://github.com/immontilla/file-uploading-web-app.git

FROM maven:alpine as build
WORKDIR /app
COPY --from=clone /app/file-uploading-web-app /app
RUN mvn -DskipTests=true clean install && cp target/secure-upload-1.0.0.jar app.jar

FROM openjdk:8-jdk-alpine
WORKDIR /app
COPY --from=build /app/app.jar /app
EXPOSE 8090
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","app.jar"]

First, you have to start the ClamAV container, otherwise, all the files will be rejected because they can't be scanned for virus.

docker run -d --name av mkodockx/docker-clamav  

Then, you can start the file uploader container.

docker run -d --link av:clamavd --volume csv-files:/tmp/safe --tmpfs /tmp/unsafe -p 8090:8090 --name secure-csv-uploader immontilla/secure-file-uploader

Finally, you can check the functionality trying to upload some files at http://localhost:8090/.

Commands to remember

• To list the docker volumes

docker volume ls

• To see information about the volume csv-files - including its path on the host machine.

docker volume inspect csv-files

• To list the uploaded files

csvFilesPath=$(docker volume inspect csv-files | grep -oP '(?<="Mountpoint": ")[^"]*')

sudo ls -l $csvFilesPath

Automated Builds have several advantages:

• Images built in this way are built exactly as specified.
• The Dockerfile is available to anyone with access to your Docker Hub repository.
• Your repository is kept up-to-date with code changes automatically.

-- Configure automated builds on Docker Hub https://docs.docker.com/docker-hub/builds/

I had a personal project at Github named file-uploading-web-app. It is a web application to upload CSV files in a secure way. The code could be easily adapted to accept any other file types.

I pushed its image on Docker Hub using a maven plugin following this post. However, as I wanted to build the image automatically, I have to create a new Github repository linked to Dockerhub. Doing so, every change I make in https://github.com/immontilla/secure-file-uploader will trigger an automated build task on https://hub.docker.com/r/immontilla/secure-file-uploader/.

This is the Dockerfile:

FROM alpine/git as clone
LABEL maintainer="Iván Mauricio Montilla Figueroa"
WORKDIR /app
RUN git clone --progress https://github.com/immontilla/file-uploading-web-app.git

FROM maven:alpine as build
WORKDIR /app
COPY --from=clone /app/file-uploading-web-app /app
RUN mvn -DskipTests=true clean install && cp target/secure-upload-1.0.0.jar app.jar

FROM openjdk:8-jdk-alpine
WORKDIR /app
COPY --from=build /app/app.jar /app
RUN mkdir /tmp/safe && mkdir /tmp/unsafe
VOLUME /tmp
EXPOSE 8090
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","app.jar"]

Commands to remember

• To gain access to the console

docker exec -it secure-csv-uploader sh

• To list the uploaded files

csvFilesPath=$(docker volume inspect csv-files | grep -oP '(?<="Mountpoint": ")[^"]*')

sudo ls -l $csvFilesPath

• To see the secure-csv-uploader log

docker logs secure-csv-uploader

Last week, I pushed docker-mitreid-oidc-server at Docker Hub. It has been pulled 50 times so far. I have received some comments demanding me to fix the simple-web-app log-in bug, and also, customize the Tomcat server default page with links to the MITREid OpenID Connect administration server and its protected client web app. Additionally, I am not satisfied with the image file size because it is larger (488Mb) than I was expecting (160 Mb).

Here you will find out what I did to fix the simple-web-app log-in bug, replace the Tomcat server default page, and reduce the image file size from 488 to 212 Mb.

simple-web-app log-in bug

I have decided to stop cloning the MITREid repositories directly in the Dockerfile, I am going to fork and clone them from my own repositories instead. Doing so, I can fix any bug with no delay. However, I have to keep an eye on the MITREid repositories updates.

Tomcat server default page

I just need to remove the ROOT folder content at /usr/local/tomcat/webapps/ROOT/ path, and copy an index.html from my GitHub repo. Doing this, I can update the html file whenever I want.

Reduce image file size

I found a guide titled Multi-stage builds at Docker docs. There you will read:

With multi-stage builds, you use multiple FROM statements in your Dockerfile. Each FROM instruction can use a different base, and each of them begins a new stage of the build. You can selectively copy artifacts from one stage to another, leaving behind everything you don’t want in the final image.

-- Use multi-stage builds

By default, the stages are not named, and you refer to them by their integer number, starting with 0 for the first FROM instruction. However, you can name your stages, by adding an as to the FROM instruction.

-- Name your build stages

Basically, I divided the Dockerfile in 3 stages:

1.- Clone

FROM alpine/git as clone 
LABEL maintainer="Iván Mauricio Montilla Figueroa"
WORKDIR /app
RUN mkdir srv && cd srv && \ 
git clone --progress https://github.com/immontilla/OpenID-Connect-Java-Spring-Server.git && \ cd .. && mkdir web && cd web && \ git clone --progress https://github.com/immontilla/simple-web-app.git

2.- Build

FROM maven:alpine as build WORKDIR /app
COPY --from=clone /app/srv/OpenID-Connect-Java-Spring-Server /app/srv
RUN cd srv && mvn -Dmaven.javadoc.skip=true -DskipTests clean install && \
mkdir ../WARs && mv openid-connect-server-webapp/target/openid-connect-server-webapp.war ../WARs
COPY --from=clone /app/web/simple-web-app /app/web
RUN cd web && mvn -DskipTests clean install && mv target/simple-web-app.war ../WARs

3.- Deploy

FROM tomcat:alpine WORKDIR /app
COPY --from=build /app/WARs /app
RUN apk add --no-cache --update openssl && \
cp *.war /usr/local/tomcat/webapps/ && \
rm -rf /usr/local/tomcat/webapps/ROOT/WEB-INF/ && \
rm /usr/local/tomcat/webapps/ROOT/* && \
cd /usr/local/tomcat/webapps/ROOT/ && \
wget https://raw.githubusercontent.com/immontilla/docker-mitreid-oidc-server/master/index.html

You can see the full Dockerfile at immontilla/docker-mitreid-oidc-server/~/dockerfile/

The final result is a 212 Mb image. It is not as small as I originally thought. The reason is, I was not taking in count the uncompressed web app folders sizes.

Let's check the result:

Tomcat:alpine size +
openid-connect-server-webapp.war size +
openid-connect-server-webapp folder size +
simple-web-app.war size +
simple-web-app folder size

113 Mb + 28.3 Mb + 33.4 Mb + 17.9 Mb + 21.1 Mb = 213.7 Mb

If you like to check this result by yourself:

Access the console machine

docker exec -it oidc-srv bash

Get every file or folder size:

du -sh /usr/local/tomcat/webapps/openid-connect-server-webapp.war

du -sh /usr/local/tomcat/webapps/openid-connect-server-webapp/

du -sh /usr/local/tomcat/webapps/simple-web-app.war

du -sh /usr/local/tomcat/webapps/simple-web-app/

and sum them.

Lessons learned along the way

• The small image file size, the better.
  
  Dividing the image building process in stages reduce its file size. In the final image there is no any java library, compiler files and/or folders and none build/compile utility at all. It is just Tomcat and two web applications.
• Latest is not always the most recent version. Usually is the stable version.
  
  This is why immontilla/docker-mitreid-oidc-server base image is not tomcat:9.0.1-alpine anymore. In fact, that version is no longer available at library/tomcat/, it was replaced by the 9.0.2. I chose tomcat:alpine as base image because its version is 8, which is the current Tomcat stable version.