Skip to content

Feed aggregator

Stateful Containers on Kubernetes using Persistent Volume and Amazon EBS

Miles to go ... - Arun Gupta - 2 hours 21 min ago

This blog will show how to create stateful containers in Kubernetes using Amazon EBS.

Couchbase is a stateful container. This means that state of the container needs to be carried with it. In Kubernetes, the smallest atomic unit of running a container is a pod. So a Couchbase container will run as a pod. And by default, all data stored in Couchbase is stored on the same host.

stateful containers

This figure is originally explained in Kubernetes Cluster on Amazon and Expose Couchbase Service. In addition, this figure shows storage local to the host.

Pods are ephemeral and may be restarted on a different host. A Kubernetes Volume outlives any containers that run within the pod, and data is preserved across container restarts. However the volume will cease to exist when a pod ceases to exist. This is solved by Persistent Volumes that provide persistent, cluster-scoped storage for applications that require long lived data.

Creating and using a persistent volume is a three step process:

  1. Provision: Administrator provision a networked storage in the cluster, such as AWS ElasticBlockStore volumes. This is called as PersistentVolume.
  2. Request storage: User requests storage for pods by using claims. Claims can specify levels of resources (CPU and memory), specific sizes and access modes (e.g. can be mounted once read/write or many times write only). This is called as PersistentVolumeClaim.
  3. Use claim: Claims are mounted as volumes and used in pods for storage.

Specifically, this blog will show how to use an AWS ElasticBlockStore as PersistentVolume, create a PersistentVolumeClaim, and then claim it in a pod.

stateful containers

Complete source code for this blog is at:

Provision AWS Elastic Block Storage

Following restrictions need to be met if Amazon ElasticBlockStorage is used as a PersistentVolume with Kubernetes:

  • the nodes on which pods are running must be AWS EC2 instances
  • those instances need to be in the same region and availability-zone as the EBS volume
  • EBS only supports a single EC2 instance mounting a volume

Create an AWS Elastic Block Storage:

aws ec2 create-volume --region us-west-2 --availability-zone us-west-2a --size 5 --volume-type gp2

The region us-west-2 region and us-west-2a availability zone is used here. And so Kubernetes cluster need to start in the same region and availability zone as well.

This shows the output as:

    "AvailabilityZone": "us-west-2a", 
    "Encrypted": false, 
    "VolumeType": "gp2", 
    "VolumeId": "vol-47f59cce", 
    "State": "creating", 
    "Iops": 100, 
    "SnapshotId": "", 
    "CreateTime": "2016-07-29T21:57:43.343Z", 
    "Size": 5

Check if the volume is available as:

aws --region us-west-2 ec2 describe-volumes --volume-id vol-47f59cce

It shows the output as:

    "Volumes": [
            "AvailabilityZone": "us-west-2a", 
            "Attachments": [], 
            "Encrypted": false, 
            "VolumeType": "gp2", 
            "VolumeId": "vol-47f59cce", 
            "State": "available", 
            "Iops": 100, 
            "SnapshotId": "", 
            "CreateTime": "2016-07-29T21:57:43.343Z", 
            "Size": 5

Note the unique identifier for the volume in VolumeId attribute. You can also verify the EBS block in AWS Console:


Start Kubernetes Cluster

Download Kubernetes 1.3.3, untar it and start the cluster on Amazon:

KUBE_AWS_ZONE=us-west-2a NODE_SIZE=m3.large NUM_NODES=3 ./kubernetes/cluster/

Three points to note here:

  • Zone in which the cluster is started is explicitly set to us-west-1a. This matches the zone where EBS storage volume was created.
  • By default, each node size is m3.medium. Here is is set to m3.large.
  • By default, 1 master and 4 worker nodes are created. Here only 3 worker nodes are created.

This will show the output as:

... Starting cluster in us-west-2a using provider aws
... calling verify-prereqs
... calling kube-up
Starting cluster using os distro: jessie
Uploading to Amazon S3
+++ Staging server tars to S3 Storage: kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel
upload: ../../../../../var/folders/81/ttv4n16x7p390cttrm_675y00000gn/T/kubernetes.XXXXXX.ISohbaGM/s3/bootstrap-script to s3://kubernetes-staging-0eaf81fbc51209dd47c13b6d8b424149/devel/bootstrap-script
Uploaded server tars:
INSTANCEPROFILE	arn:aws:iam::598307997273:instance-profile/kubernetes-master	2016-07-29T15:13:35Z	AIPAJF3XKLNKOXOTQOCTkubernetes-master	/
ROLES	arn:aws:iam::598307997273:role/kubernetes-master	2016-07-29T15:13:33Z	/	AROAI3Q2KFBD5PCKRXCRM	kubernetes-master
STATEMENT	sts:AssumeRole	Allow
INSTANCEPROFILE	arn:aws:iam::598307997273:instance-profile/kubernetes-minion	2016-07-29T15:13:39Z	AIPAIYSH5DJA4UPQIP4Bkubernetes-minion	/
ROLES	arn:aws:iam::598307997273:role/kubernetes-minion	2016-07-29T15:13:37Z	/	AROAIQ57MPQYSHRPQCT2Q	kubernetes-minion
STATEMENT	sts:AssumeRole	Allow
Using SSH key with (AWS) fingerprint: SHA256:dX/5wpWuUxYar2NFuGwiZuRiydiZCyx4DGoZ5/jL/j8
Creating vpc.
Adding tag to vpc-fa3d6c9e: Name=kubernetes-vpc
Adding tag to vpc-fa3d6c9e: KubernetesCluster=kubernetes
Using VPC vpc-fa3d6c9e
Adding tag to dopt-3aad625e: Name=kubernetes-dhcp-option-set
Adding tag to dopt-3aad625e: KubernetesCluster=kubernetes
Using DHCP option set dopt-3aad625e
Creating subnet.
Adding tag to subnet-e11f5985: KubernetesCluster=kubernetes
Using subnet subnet-e11f5985
Creating Internet Gateway.
Using Internet Gateway igw-5c748f38
Associating route table.
Creating route table
Adding tag to rtb-84fcf1e0: KubernetesCluster=kubernetes
Associating route table rtb-84fcf1e0 to subnet subnet-e11f5985
Adding route to route table rtb-84fcf1e0
Using Route Table rtb-84fcf1e0
Creating master security group.
Creating security group kubernetes-master-kubernetes.
Adding tag to sg-91590bf7: KubernetesCluster=kubernetes
Creating minion security group.
Creating security group kubernetes-minion-kubernetes.
Adding tag to sg-9d590bfb: KubernetesCluster=kubernetes
Using master security group: kubernetes-master-kubernetes sg-91590bf7
Using minion security group: kubernetes-minion-kubernetes sg-9d590bfb
Creating master disk: size 20GB, type gp2
Adding tag to vol-def79e57: Name=kubernetes-master-pd
Adding tag to vol-def79e57: KubernetesCluster=kubernetes
Allocated Elastic IP for master:
Adding tag to vol-def79e57:
Generating certs for alternate-names: IP:,IP:,IP:,DNS:kubernetes,DNS:kubernetes.default,DNS:kubernetes.default.svc,DNS:kubernetes.default.svc.cluster.local,DNS:kubernetes-master
Starting Master
Adding tag to i-5a7cebf5: Name=kubernetes-master
Adding tag to i-5a7cebf5: Role=kubernetes-master
Adding tag to i-5a7cebf5: KubernetesCluster=kubernetes
Waiting for master to be ready
Attempt 1 to check for master nodeWaiting for instance i-5a7cebf5 to be running (currently pending)
Sleeping for 3 seconds...
Waiting for instance i-5a7cebf5 to be running (currently pending)
Sleeping for 3 seconds...
Waiting for instance i-5a7cebf5 to be running (currently pending)
Sleeping for 3 seconds...
Waiting for instance i-5a7cebf5 to be running (currently pending)
Sleeping for 3 seconds...
 [master running]
Attaching IP to instance i-5a7cebf5
Attaching persistent data volume (vol-def79e57) to master
2016-07-29T22:00:36.909Z	/dev/sdb	i-5a7cebf5	attaching	vol-def79e57
cluster "aws_kubernetes" set.
user "aws_kubernetes" set.
context "aws_kubernetes" set.
switched to context "aws_kubernetes".
user "aws_kubernetes-basic-auth" set.
Wrote config for aws_kubernetes to /Users/arungupta/.kube/config
Creating minion configuration
Creating autoscaling group
 0 minions started; waiting
 0 minions started; waiting
 0 minions started; waiting
 0 minions started; waiting
 3 minions started; ready
Waiting for cluster initialization.

  This will continually check to see if the API for kubernetes is reachable.
  This might loop forever if there was some uncaught error during start

..........................................................................................................................................................................................................Kubernetes cluster created.
Sanity checking cluster...
Attempt 1 to check Docker on node @ ...not working yet
Attempt 2 to check Docker on node @ ...not working yet
Attempt 3 to check Docker on node @ ...working
Attempt 1 to check Docker on node @ ...working
Attempt 1 to check Docker on node @ ...working

Kubernetes cluster is running.  The master is running at:

The user name and password to use is located in /Users/arungupta/.kube/config.

... calling validate-cluster
Found 3 node(s).
NAME                                        STATUS    AGE   Ready     1m   Ready     1m   Ready     1m
Validate output:
NAME                 STATUS    MESSAGE              ERROR
controller-manager   Healthy   ok                   
scheduler            Healthy   ok                   
etcd-0               Healthy   {"health": "true"}   
etcd-1               Healthy   {"health": "true"}   
Cluster validation succeeded
Done, listing cluster services:

Kubernetes master is running at
Elasticsearch is running at
Heapster is running at
Kibana is running at
KubeDNS is running at
kubernetes-dashboard is running at
Grafana is running at
InfluxDB is running at

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

Read more details about starting a Kubernetes cluster on Amazon.

Couchbase Pod w/o Persistent Storage

Let’s create a Couchbase pod without persistent storage. This means that if the pod is rescheduled on a different host then it will not have access to the data created on it.

Here are quick steps to run a Couchbase pod and expose it outside the cluster: run couchbase --image=arungupta/couchbase expose deployment couchbase --target-port=8091 --port=8091 --type=LoadBalancer describe svc couchbase

Read more details at Kubernetes cluster at Amazon.

The last command shows the ingress load balancer address. Access Couchbase Web Console at <ip>:8091.


Login to the console using Administrator login and password password.

The main page of Couchbase Web Console shows up:


A default travel-sample bucket is already created by arungupta/couchbase image. This bucket is shown in the Data Buckets tab:


Click on Create New Data Bucket button to create a new data bucket. Give it a name k8s, take all the defaults, and click on Create button to create the bucket:


Created bucket is shown in the Data Buckets tab:


Check status of the pod: get po
NAME                         READY     STATUS    RESTARTS   AGE
couchbase-2646907196-memz2   1/1       Running   0          53m

Delete the pod: delete po couchbase-2646907196-memz2
pod "couchbase-2646907196-memz2" deleted

Watch the new pod being created: get -w po
NAME                         READY     STATUS        RESTARTS   AGE
couchbase-2646907196-memz2   1/1       Terminating   0          53m
couchbase-2646907196-wo6ve   1/1       Running       0          3s

Access the Web Console again and see that the bucket does not exist:


Let’s clean up the resources created: delete svc couchbase delete deployment couchbase

Couchbase Pod with Persistent Storage

Now, lets expose a Couchbase pod with persistent storage. As discussed above, lets create a PersistentVolume and claim the volume.

Request storage

Like any other Kubernetes resources, a persistent volume is created by using a resource description file:

kind: PersistentVolume
apiVersion: v1
  name: couchbase-pv
    type: amazonEBS
    storage: 5Gi
    - ReadWriteOnce
    volumeID: vol-47f59cce
    fsType: ext4

The important pieces of information here are:

  • Creating a storage of 5 GB
  • Storage can be mounted by only one node for reading/writing
  • specifies the volume id created earlier

Read more details about definition of this file at

This file is available at:

The volume itself can be created as:

kubectl create -f couchbase-pv.yml

and shows the output:

persistentvolume "couchbase-pv" created

Use claim

A PersistentVolumeClaim can be created using this resource file:

kind: PersistentVolumeClaim
apiVersion: v1
  name: couchbase-pvc
    type: amazonEBS
    - ReadWriteOnce
      storage: 5Gi

In our case, both PersistentVolume and PersistentVolumeClaim are 5 GB but they don’t have to be.

Read more details about definition of this file at

This file is at

The claim can be created as:

kubectl create -f couchbase-pvc.yml

and shows the output:

persistentvolumeclaim "couchbase-pvc" created

Create RC with Persistent Volume Claim

Create a Couchbase Replication Controller using this resource file:

apiVersion: v1
kind: ReplicationController
  name: couchbase
  replicas: 1
      name: couchbase-rc-pod
        name: couchbase-rc-pod
        context: couchbase-pv
      - name: couchbase-rc-pod
        image: arungupta/couchbase
        - mountPath: "/opt/couchbase/var"
          name: mypd
        - containerPort: 8091
        - containerPort: 8092
        - containerPort: 8093
        - containerPort: 11210
      - name: mypd
          claimName: couchbase-pvc

Key parts here are:

  • Resource defines a Replication Controller using arungupta/couchbase Docker image
  • volumeMounts define which volumes are going to be mounted. /opt/couchbase/var is the directory where Couchbase stores all the data.
  • volumes define different volumes that can be used in this RC definition

Create the RC as:

kubectl create -f couchbase-rc.yml

and shows the output:

replicationcontroller "couchbase" created

Check for pod as get -w po to see: get -w po
NAME              READY     STATUS              RESTARTS   AGE
couchbase-jx3fn   0/1       ContainerCreating   0          3s
NAME              READY     STATUS    RESTARTS   AGE
couchbase-jx3fn   1/1       Running   0          20s

Expose RC as a service: expose rc couchbase --target-port=8091 --port=809--type=LoadBalancer
service "couchbase" exposed

Get all the services: get svc
couchbase   a6179426155e2...   8091/TCP   19s
kubernetes      <none>             443/TCP    1h

Describe the service as describe svc couchbase to see:

Name:			couchbase
Namespace:		default
Labels:			context=couchbase-pv
Selector:		context=couchbase-pv,name=couchbase-pod
Type:			LoadBalancer
LoadBalancer Ingress:
Port:			<unset>	8091/TCP
NodePort:		<unset>	31636/TCP
Session Affinity:	None
  FirstSeen	LastSeen	Count	From			SubobjectPath	Type		Reason			Message
  ---------	--------	-----	----			-------------	--------	------			-------
  31s		31s		1	{service-controller }			Normal		CreatingLoadBalancer	Creating load balancer
  29s		29s		1	{service-controller }			Normal		CreatedLoadBalancer	Created load balancer

Wait for ~3 mins for the load balancer to settle. Access the Couchbase Web Console at <ingress-lb>:8091. Once again, only travel-sample bucket exists. This is created by arungupta/couchbase image used in the RC definition.

Show Stateful Containers

Lets create a new bucket. Give it a name kubernetes-pv, take all defaults and click on Create button to create the bucket.


The bucket now shows up in the console:


Terminate Couchbase pod and see the state getting restored.

Get the pods again: get po
NAME              READY     STATUS    RESTARTS   AGE
couchbase-jx3fn   1/1       Running   0          7m

Delete the pod: delete po couchbase-jx3fn
pod "couchbase-jx3fn" deleted

Pod gets recreated: get -w po
NAME              READY     STATUS              RESTARTS   AGE
couchbase-jx3fn   1/1       Terminating         0          8m
couchbase-qq6wu   0/1       ContainerCreating   0          4s
NAME              READY     STATUS    RESTARTS   AGE
couchbase-qq6wu   1/1       Running   0          5s

And now when you access the Couchbase Web Console, the earlier created bucket still exists:

kubernetes-pv-couchbase-amazon-kubernetes-pv-bucket-still-thereThat’s because the data was stored in the backing EBS storage.

Cleanup Kubernetes Cluster

Shutdown Kubernetes cluster:

KUBE_AWS_ZONE=us-west-2a NODE_SIZE=m3.large NUM_NODES=3 ./kubernetes/cluster/ 
Bringing down cluster using provider: aws
Deleting ELBs in: vpc-fa3d6c9e
Waiting for ELBs to be deleted
All ELBs deleted
Deleting instances in VPC: vpc-fa3d6c9e
Deleting auto-scaling group: kubernetes-minion-group-us-west-2a
Deleting auto-scaling launch configuration: kubernetes-minion-group-us-west-2a
Deleting auto-scaling group: kubernetes-minion-group-us-west-2a
Deleting auto-scaling group: kubernetes-minion-group-us-west-2a
Waiting for instances to be deleted
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
Waiting for instance i-8b7aed24 to be terminated (currently shutting-down)
Sleeping for 3 seconds...
All instances deleted
Releasing Elastic IP:
Deleting volume vol-def79e57
Cleaning up resources in VPC: vpc-fa3d6c9e
Cleaning up security group: sg-91590bf7
Cleaning up security group: sg-9d590bfb
Cleaning up security group: sg-e97b298f
Deleting security group: sg-91590bf7
Deleting security group: sg-9d590bfb
Deleting security group: sg-e97b298f
Deleting VPC: vpc-fa3d6c9e

And detach the volume:

aws ec2 delete-volume --region us-west-2 --volume-id vol-47f59cce

Complete source code for this blog is at:



The post Stateful Containers on Kubernetes using Persistent Volume and Amazon EBS appeared first on Miles to go 3.0 ....

Related posts:

  1. Kubernetes Cluster on Amazon and Expose Couchbase Service
  2. Kubernetes Cluster on Google Cloud and Expose Couchbase Service
  3. Kubernetes Namespaces, Resource Quota, and Limits for QoS in Cluster

Categories: Companies

Spring Data Release Train Ingalls M1 Released

On behalf of the Spring Data team, I’m happy to announce the first milestone of the Ingalls release train. The release ships 230 tickets fixed! The most noteworthy new features are:

  • Use of method handles for property access in conversion subsystem (Commons, MongoDB).
  • Upgrade to Cassandra 3.0 for Spring Data Cassandra.
  • Support for declarative query methods for Cassandra repositories.
  • Support for Redis geo commands.
  • Any-match mode for query-by-example.
  • Support for XML and JSON based projections for REST payloads (see the example for details)

Find a curated change log in our release train wiki or skim through a full list of changes in JIRA.

Categories: Communities

Government Asks: What’s in Your Software?

Sonatype Blog - Fri, 07/29/2016 - 21:56
U.S. Government pays closer attention to software components Multiple agencies across the U.S. government are paying closer attention to the software they are buying.  More specifically, they want to know what open souce and third party components were used to build the software...

To read more, visit our blog at
Categories: Companies

5 Up-and-Coming Programming Languages to Know About

Staying current in the programming field can sometimes make you feel like the Red Queen in “Alice Through the Looking-Glass.” She said, “It takes all the running you can do to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!”

You’re a master at Ruby on Rails? Great, but how are you with statistical analysis in R? Want to work at Google? Forget Python and start building channels in Go.

Categories: Communities

This Week in Spring: Spring Cloud M1 Available

Welcome to another installation of This Week in Spring! This week I’m mostly in San Francisco and Saint Louis, busily preparing for the big event next week!

This is my favorite time of year! As we lead to SpringOne Platform, there’s so much good stuff being released that one can hardly keep up! I am really looking forward to this year’s SpringOne Platform show, coming in early August. It’s an amazing time to build applications, and SpringOne Platform is in a unique position to capture the larger discussion: why do we #devops, #cloud, #agile, or #microservice? Join the discussion August 1-4, 2016 in beautiful Las Vegas and let’s find out!

Categories: Communities

Java Web Weekly, Issue 135

baeldung - Coding and Testing Stuff - Fri, 07/29/2016 - 09:48
I just released the Master Class of "Learn Spring Security":


At the very beginning of last year, I decided to track my reading habits and share the best stuff here, on Baeldung. Haven’t missed a review since.

Here we go…

1. Spring and Java >> Reactive Programming with Spring 5.0 M1 []

Reactive support has finally been merged and is included into this cut of the 5.x version of the framework.

If you’ve been holding off from exploring it before, now’s a good time to get your feet wet.

>> Java on Steroids: 5 Super Useful JIT Optimization Techniques []

Interesting notes about what JVM compiler does to optimize the performance of our production code.

>> Java 8 Top Tips []

A good read with or without the IntelliJ tips. Of course if you’re using IntelliJ, it’s even better.

>> Oh No, I Forgot Stream::iterate! []

Quick and to the point – some nice APIs are coming along with Java 9.

>> The best way to map a @OneToOne relationship with JPA and Hibernate []

Hehe, the good ol’ one-to-one relationship.

>> Oracle Paves the Way to Standardise Command Line Options in the JDK []

A quick positive news item slotted for Java 10, among all the negative news about Java EE lately.

Also worth reading:

Webinars and presentations:

Time to upgrade:

2. Technical and Musings >> DDD Decoded – Don’t Fear Eventual Consistency []

“The only people worried about Eventual Consistency are CRUD programmers”. Priceless.

I could keep quoting, but do yourself a favor and read this one, not only for the technical aspects, but also for the laughs.

>> I wanna go fast: HTTPS’ massive speed advantage []

Quite some interesting data here, looking at the speed advantage of HTTP/2.

>> The Business-Personal Value Continuum []

The writeup explores a tough topic to pin down – the way we approach our work and the balance between our natural need towards craftsmanship and actually putting out work into the world.

I personally strive to keep a healthy tension between the two and, as Sean says – ship when it’s 90% perfect.

>> Logs for SEO []

A fun read for anyone running any kind of site and dabbling in SEO.

>> 12 great months for Thoughts on Java and some big changes ahead []

I always enjoy this kind of post. I think there’s a lot here to unpack and reading through the thought process some else has about a tough life choice is quite helpful. Especially when it deals with such important decisions as – should I quit my job?

Also worth reading:

3. Comics

And my favorite Dilberts of the week:

>> Mordac, the preventer of information services [] >> What do you do for a living? [] >> This design would be inefficient [] 4. Pick of the Week

My talk from Spring I/O 2016 came out – if you’re working on REST API and Hypermedia with Spring, let me know what you think:

>> Get HATEOAS and Hypermedia right with Spring [] I just released the Master Class of "Learn Spring Security" Course:


Categories: Blogs

ActiveRecord Is Even Worse Than ORM

You probably remember what I think about ORM, a very popular design pattern. In a nutshell, it encourages us to turn objects into DTOs, which are anemic, passive, and not objects at all. The consequences are usually dramatic — the entire programming paradigm shifts from object-oriented to procedural. I've tried to explain this at a JPoint and JEEConf this year. After each talk, a few people told me that what I'm suggesting is called ActiveRecord or Repository patterns.

Moreover, they claimed that ActiveRecord actually solves the problem I've found in ORM. They said I should explain in my talks that what I'm offering (SQL-speaking objects) already exists and has a name: ActiveRecord.

Categories: Communities

Anticipated Impact of Java 9

To gather insights on the state of the Java ecosystem today for DZone's Java Ecosystem research guide to be published in September, we spoke with 15 executives who are familiar with the Java ecosystem.

Here’s who we talked to:

Categories: Communities

Two Kinds of Simplicity

I was reading Niklaus Wirth's On the Design of Programming Languages and was struck by his discussion of simplicity. It appeared to me to apply to a number of concepts in architecture and design beyond just programming languages, and even to explain why we so often disagree about design choices.

One of the key insights of his paper is that there are multiple kinds of simplicity. He gives the example of generalizing all of the different data types into untyped values. He points out that this simplicity through generality makes it easier to write compilers, but shifts the onus to the programmer to make sure that type-less values are used correctly. So it's really a tradeoff of one kind of simplicity for another.

Categories: Communities

Simple Form Login Page With Apache Sling

In this post we will show a simple example on how to create a login page with Apache Sling and define which paths will ask for the user to login to the site using our login page.

In order to execute this, we need to have the Sling Lauchpad running on our machine. For this we can download the binary from the Apache Sling website in Downloads Section. If you have Docker installed on your machine you can use the Sling docker image.

Categories: Communities

Quick Guide to Spring MVC with Velocity

baeldung - Coding and Testing Stuff - Thu, 07/28/2016 - 10:47
The Master Class of "Learn Spring Security" is out:


1. Introduction

Velocity is a template engine from the Apache Software Foundation that can work with normal text files, SQL, XML, Java code and many other types.

In this article we’re going to focus on utilizing Velocity with a typical Spring MVC web application.

2. Maven Dependencies

Let’s start by enabling the Velocity support – with the following dependencies:


The newest versions of both can be found here: velocity and velocity-tools.

3. Configuration 3.1. Web Config

If we don’t want to use an web.xml, let’s configure our web project using Java and an initializer:

public class MainWebAppInitializer implements WebApplicationInitializer {

    public void onStartup(ServletContext sc) throws ServletException {
        AnnotationConfigWebApplicationContext root = new AnnotationConfigWebApplicationContext();

        sc.addListener(new ContextLoaderListener(root));

        ServletRegistration.Dynamic appServlet = 
          sc.addServlet("mvc", new DispatcherServlet(new GenericWebApplicationContext()));

Alternatively, we can of course use the traditional web.xml:

<web-app ...>
    <display-name>Spring MVC Velocity</display-name>


Notice that we mapped our servlet on the “/*” path.

3.2. Spring Config

Let’s now go over a simple Spring configuration – again, starting with Java:

@ComponentScan(basePackages= {
  "com.baeldung.mvc.velocity.service" }) 
public class WebConfig extends WebMvcConfigurerAdapter {

    public void addResourceHandlers(ResourceHandlerRegistry registry) {
    public void configureDefaultServletHandling(DefaultServletHandlerConfigurer configurer) {

    public ViewResolver viewResolver() {
        VelocityLayoutViewResolver bean = new VelocityLayoutViewResolver();
        return bean;
    public VelocityConfigurer velocityConfig() {
        VelocityConfigurer velocityConfigurer = new VelocityConfigurer();
        return velocityConfigurer;

And let’s also have a quick look at the XML version of the configuration:

<beans ...>
    <context:component-scan base-package="com.baeldung.mvc.velocity.*" />
    <context:annotation-config /> 
    <bean id="velocityConfig" 
        <property name="resourceLoaderPath">
    <bean id="viewResolver"
        <property name="cache" value="true" />
        <property name="prefix" value="/WEB-INF/views/" />
        <property name="layoutUrl" value="/WEB-INF/layouts/layout.vm" />
        <property name="suffix" value=".vm" />

Here we are telling Spring where to look for annotated bean definitions:

<context:component-scan base-package="com.baeldung.mvc.velocity.*" />

We are indicating we are going to use annotation-driven configuration in our project with the following line: 

<context:annotation-config />

By creating “velocityConfig” and “viewResolver” beans we are telling VelocityConfigurer where to look for templates, and VelocityLayoutViewResolver where to find views and layouts.

4. Velocity Templates

Finally, let’s create our templates – starting with a common header:

<div style="...">
    <div style="float: left">
        <h1>Our tutorials</h1>

and footer:

<div style="...">

And let’s define a common layout for our site where we are going to use above fragments with parse in the following code:

        <title>Spring & Velocity</title>  
            <!-- View index.vm is inserted here -->

You can check that $screen_content variable has the content of the pages.

Finally, we’ll create a template for the main content:

<h2>Tutorials list</h2>
<table border="1">
        <th>Tutorial Id</th>
        <th>Tutorial Title</th>
        <th>Tutorial Description</th>
        <th>Tutorial Author</th>
    #foreach($tut in $tutorials)
5. Controller Side

We have created a simple controller which returns a list of tutorials as content for our layout to be populated with:

public class MainController {
    private ITutorialsService tutService;

    @RequestMapping(value ="/", method = RequestMethod.GET)
    public String defaultPage() {
        return "index";

    @RequestMapping(value ="/list", method = RequestMethod.GET)
    public String listTutorialsPage(Model model) { 
        List<Tutorial> list = tutService.listTutorials();
        model.addAttribute("tutorials", list);
        return "index";

Finally, we can access this simple example locally – for example at: localhost:8080/spring-mvc-velocity/

6. Conclusion

In this simple tutorial, we have configured Spring MVC web application with Velocity template engine.

The full sample code for this tutorial can be found in our GitHub repository.

The Master Class of "Learn Spring Security" is out:


Categories: Blogs

Signing Java Code With Certum Open Source Certificates

Recent versions of Java have tightened the security around code that is run from the web, which is good news for end users, but a pain for developers. These new requirements forced users of a new web start Java project that I have worked on to manually add exceptions for the location of the JAR files being downloaded. This sounds like a simple thing to do, but turned out to be quite cumbersome, so I decided to get a real certificate.

Certum offers reasonably priced certificates for those working on open source projects. This is a great service for those who can’t justify hundreds of dollars a year on a certificate for code that they give away, and I bought one of these certificates for my own project.

Categories: Communities

Queries and Aggregations With Scala: Part 2

As we saw in part 1, it took 2+ minutes, on my laptop, to run an aggregation across 6+ million crime cases.

However, since Hazelcast is a cluster library, it only makes sense to take advantage of that. Let’s run this on a 3-node cluster. That will require a few code changes to CrimeNode:

Categories: Communities

How the Java Community Process Has Changed and Where It's Headed

To gather insights on the state of the Java ecosystem today for DZone's Java Ecosystem research guide to be published in September, we spoke with 15 executives who are familiar with the Java ecosystem.

Here’s who we talked to:

Categories: Communities

Java Code Challenge: Chemical Symbol Naming-Part One

Java Code Challenge is a new regular segment taking the best challenge from Reddit's dailyprogrammer. Things are a little different here as we're focused on Java. A working solution is not enough; we're looking for the cleanest Java code with tests. 3rd party libraries are welcome but if you can do it without it will be easier for others to comprehend.

If you can fit your solution in the comments then go for it, but preferably put your answer in GitHub and link in the comments. Next week we'll be sharing the best solutions and sharing the best code practices we see!

Categories: Communities

Pattern Implementations


The “implementation” or “examples” section of any pattern discussion holds several goals (and a few “anti-goals”):

  • Be able to put a concrete-ish example in front of people seeking such. It’s hard to understand exactly how the pattern is supposed to work without pictures or code. I am not great with graphical tools, so for me it’s easier to use code to provide that demonstration.
  • NOT to expect “ready-made code” for reuse. Patterns are not drop-in building blocks that can save you time and energy when doing your own implementation. These examples are here to demonstrate a few techniques around implementation, but attempts to re-use the code directly will probably always meet with failure at some level.
  • Demonstrate how to do certain things idiomatically within a particular language. Each language brings with it a particular idiomatic style or feature set that may shape how one might use a particular pattern. I am not an expert with all of these languages below, but part of this exercise (for me) is to learn (and document) how to exercise the idioms of a particular language using the pattern as a scaffold upon which to hang it.
  • Provide a mechanism by which to concretely compare and contrast one pattern against another. Is a Strategy really all that close to a Command? Having concrete examples of each allows for a certain amount of comparison-and-contrast, and hopefully sparks some good discussion around when to use each.
  • NOT to suggest that one language is “better” than another. Any such qualitative judgment around one language over another is entirely in the eyes of the beholder; no such judgement is intended from me, and any attempt to use this exercise as a means to judge one language more harshly than another will quickly earn this author’s scorn. Different languages chose to do things in different ways for very good reasons; if you cannot explain the reasons, then you have no business offering up the judgement.

In general, there’s a long list of languages I will use to define some example implementations of the patterns in the catalog. Note that while this isn’t an “ordered” list, meaning I will probably do implementations in a seemingly-random order, the hope is that when this is all said and done, the list of pattern implementations will range across the following:

Categories: Communities

Couchbase Connect 2016 Call for Papers

Miles to go ... - Arun Gupta - Thu, 07/28/2016 - 02:19

Couchbase Connect is an event where the best Devs and Ops mind in NoSQL get together!

Inviting all developers, architects, administrators, and CxOs to Save The Date for Couchbase Connect. This event is happening in the beautiful Bay Area and could be your excuse to visit Silicon Valley! And if you are living near by, then there is no excuse not to attend!


You will hear from best speakers in the NoSQL industry. You’ll also get an opportunity to meet Couchbase Product Managers, Engineers who build the technology, Developer Advocates, Community Champions and Experts, Executive Team of Couchbase, and many more. Who knows which discussion will help you replatform your application and get ready to meet the demands of Digital Economy!

You’ll not only learn about the some of the coolest features in Couchbase, but also get to hear about our product roadmap. There will be hands-on workshops, sessions by practitioners, quick tips, informal Birds of Feather discussions and a lot more. And of course, the hallway track will allow you to socialize with attendeesand other Couchbasers.

Couchbase Connect 2016 Call for Papers

Are you using Couchbase and would like to share your success story? We’d love to have you submit a talk.

Some of the suggested topics are …

  • Build a polyglot application with microservices and using Couchbase
  • Deploy Couchbase container using any of the orchestration frameworks like Docker Swarm, Kubernetes, Mesos, and AWS ECS
  • Using Couchbase Mobile in a creative way
  • Making Couchbase Mobile offline work for you in an interesting environment
  • Provisioning, managing, scaling, and monitoring Couchbase clusters
  • Integration with Big Data platform such as Hadoop, Spark, and Kafka
  • Any interesting tools that you’ve built around Couchbase
  • Migration from RDBMS or other NoSQL databases to Couchbase
  • Using Couchbase in production with hundreds of nodes and multiple clusters
  • Connecting edge/IoT devices using Couchbase Mobile and Server

And these are only a suggestion. Feel free to get creative and go crazy!

Submit your talks at:

Each talk is usually 40 mins long. But you can submit a lightning talk for 10 or 20 mins as well. The Call for Papers end on Sep 1, but don’t delay. Start thinking about those abstracts and submit a session today!

All speakers will be invited to a VIP dinner, get some exclusive swag, and a whole lot more.

Sessions from Connect 2015 can be viewed at

Check out some pictures from Connect 2015:

connect15.1 connect15.2 connect15.3 connect15.4 connect15.5 connect15.6 connect15.7 connect15.8 connect15.9 connect15.10

If pictures are any indication, Connect will be a complete NoSQL geekgasm. You don’t want to miss the party and all the fun!

Couchbase Connect References


The post Couchbase Connect 2016 Call for Papers appeared first on Miles to go 3.0 ....

Related posts:

  1. 20 Ways to Connect with Couchbase Community
  2. Couchbase Mobile at Xamarin Evolve
  3. Couchbase Weekly, Mar 28, 2016

Categories: Companies

Intro to Spring Security Expressions

baeldung - Coding and Testing Stuff - Wed, 07/27/2016 - 16:54
1. Introduction

In this tutorial we’ll focus on Spring Security Expressions, and of course on practical examples with these expressions.

Before looking at more complex implementations (such as ACL), it’s important to have a solid grasp on security expressions – as they can be quite flexible and powerful if used correctly.

This article is an extension of Spring Security Expressions – hasRole Example.

2. Maven Dependencies

In order to use Spring Security, you need to include the following section in your pom.xml file:


Latest version can be found here.

And a quick note – this dependency only covers Spring Security; don’t forget to add spring-core and spring-context for a full web application.

3. Configuration

First, let’s take a look at a Java configuration.

We’ll extend WebSecurityConfigurerAdapter – so that we have the option to hook into any of the extension points that base class offers:

@EnableGlobalMethodSecurity(prePostEnabled = true)
public class SecurityWithoutCsrfConfig extends WebSecurityConfigurerAdapter {

We can of course do an XML configuration as well:

<?xml version="1.0" encoding="UTF-8"?>
<beans:beans ...>
    <global-method-security pre-post-annotations="enabled"/>
4. Web Security Expressions

Now, let’s start looking at the security expressions:

  • hasRole, hasAnyRole
  • hasAuthority, hasAnyAuthority
  • permitAll, denyAll
  • isAnonymous, isRememberMe, isAuthenticated, isFullyAuthenticated
  • principal, authentication
  • hasPermission

And let’s now go over each of these in detail.

4.1. hasRole, hasAnyRole

These expressions are responsible for defining the access control or authorization to specific URLs or methods in your application.

Let’s look at the example:

protected void configure(final HttpSecurity http) throws Exception {

In this example we specify access to all links starting with /auth/ restricted to users that are logged in with role USER or role ADMIN. Moreover, to access links starting with /auth/admin/ we need to have ADMIN role in the system.

The same configuration might be achieved in XML file, by writing:

    <intercept-url pattern="/auth/admin/*" access="hasRole('ADMIN')"/>
    <intercept-url pattern="/auth/*" access="hasAnyRole('ADMIN','USER')"/>
4.2. hasAuthority, hasAnyAuthority

Roles and authorities are similar in Spring.

The main difference is that, roles have special semantics – starting with Spring Security 4, the ‘ROLE_‘ prefix is automatically added (if it’s not already there) by any role related method.

So hasAuthority(‘ROLE_ADMIN’) is similar to hasRole(‘ADMIN’) because the ‘ROLE_‘ prefix gets added automatically.

But the good thing about using authorities is that we don’t have to use the ROLE_ prefix at all.

Here’s a quick example where we’re defining users with specific authorities:

protected void configure(AuthenticationManagerBuilder auth) throws Exception {

We can then of course use these authorities expressions:

protected void configure(final HttpSecurity http) throws Exception {
    .antMatchers("/auth/*").hasAnyAuthority("ADMIN", "USER")

As we can see – we’re not mentioning roles at all here.

Finally – we can of course achieve the same functionality using XML configuration as well:

            <user name="user1" password="user1Pass" authorities="ROLE_USER"/>
            <user name="admin" password="adminPass" authorities="ROLE_ADMIN"/>


    <intercept-url pattern="/auth/admin/*" access="hasAuthority('ADMIN')"/>
    <intercept-url pattern="/auth/*" access="hasAnyAuthority('ADMIN','USER')"/>
4.3. permitAll, denyAll

Those two annotations are also quite straightforward. We either may permit access to some URL in our service or we may deny access.

Let’s have a look at the example:


With this config, we will authorize all users (both anonymous and logged in) to access the page starting with ‘/’ (for example, our homepage).

We can also deny access to our entire URL space:


And again, the same config can be done with an XML config as well:

<http auto-config="true" use-expressions="true">
    <intercept-url access="permitAll" pattern="/*" /> <!-- Choose only one -->
    <intercept-url access="denyAll" pattern="/*" /> <!-- Choose only one -->
4.4. isAnonymous, isRememberMe, isAuthenticated, isFullyAuthenticated

In this subsection we focus on expressions related to login status of the user. Let’s start with user that didn’t log in to our page. By specifying following in Java config, we enable all unauthorized users to access our main page:


The same in XML config:

    <intercept-url pattern="/*" access="isAnonymous()"/>

If we want to secure the website that everybody who uses it will be required to login, we need to use isAuthenticated() method:


or XML version:

    <intercept-url pattern="/*" access="isAuthenticated()"/>

Moreover, we have two additional expressions, isRememberMe() and isFullyAuthenticated(). Through the use of cookies, Spring enables remember-me capabilities so there is no need to log into the system each time. You can read more about Remember Me here.

In order to give the access to users that were logged in only by remember me function, we may use this:


or XML version:

    <intercept-url pattern="*" access="isRememberMe()"/>

Finally, some parts of our services require the user to be authenticated again even if the user is already logged in. For example, user wants to change settings or payment information; it’s of course good practice to ask for manual authentication in the more sensitive areas of the system.

In order to do that, we may specify isFullyAuthenticated(), which returns true if the user is not an anonymous or a remember-me user:


or the XML version:

    <intercept-url pattern="*" access="isFullyAuthenticated()"/>
4.5. principal, authentication

These expressions allow accessing the principal object representing the current authorized (or anonymous) user and the current Authentication object from the SecurityContext, respectively.

We can, for example, use principal to load a user’s email, avatar, or any other data that is accessible in the logged in user.

And authentication provides information about the full Authentication object, along with its granted authorities.

Both are described in further detail in the following article: Retrieve User Information in Spring Security.

4.6. hasPermission APIs

This expression is documented and intended to bridge between the expression system and Spring Security’s ACL system, allowing us to specify authorization constraints on individual domain objects, based on abstract permissions.

Let’s have a look at an example. We have a service that allows cooperative writing articles, with a main editor, deciding which article proposed by other authors should be published.

In order to allow usage of such a service, we may create following methods with access control methods:

@PreAuthorize("hasPermission(#articleId, 'isEditor')")
public void acceptArticle(Article article) {

Only authorized user can call this method, and also user needs to have permission isEditor in the service.

We also need to remember to explicitly configure a PermissionEvaluator in our application context:

<global-method-security pre-post-annotations="enabled">
    <expression-handler ref="expressionHandler"/>

<bean id="expressionHandler"
    <property name="permissionEvaluator" ref="customInterfaceImplementation"/>

where customInterfaceImplementation will be the class that implements PermissionEvaluator. 

Of course we can also do this with Java configuration as well:

protected MethodSecurityExpressionHandler expressionHandler() {
    DefaultMethodSecurityExpressionHandler expressionHandler = 
      new DefaultMethodSecurityExpressionHandler();
    expressionHandler.setPermissionEvaluator(new CustomInterfaceImplementation());
    return expressionHandler;
5. Conclusion

This tutorial is a comprehensive introduction and guide to Spring Security Expressions.

All examples discussed here are available on the GitHub project.

Categories: Blogs

Gradle Goodness: Set VCS for IntelliJ IDEA in Build File [Snippet]

When we use the IDEA plugin in Gradle we can generate IntelliJ IDEA project files. We can customise the generated files in different ways. One of them is using a simple DSL to configure certain parts of the project file. With the DSL it is easy to set the version control system (VCS) used in our project.

In the next example build file we'll customize the generated IDEA project file and set Git as the version control system. The property is still incubating, but we can use it to have a proper configuration.

Categories: Communities

What’s New in Intellij Idea 2016.2 for Spring Developers

If you’re a heavy user of Spring and IntelliJ IDEA, you’ll find the IntelliJ IDEA 2016.2 update very interesting. Besides many important bugfixes, the update adds support for Spring 4.3 and Spring Security 4.0 features and cache abstraction, improves support for Spring MVC, and further tunes performance. Before reading about this in more detail, make sure you’ve registered for our webinar dedicated to the new release and Spring framework.

OK, now back to the update. First of all, the IDE now supports generics types as qualifiers and provides navigation for them:

Categories: Communities