CxfSerlvet OutOfMemoryError

I was working with uploading large files to my Spring application, and I encountered an OutOfMemoryError.

java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:2271)
        at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
        at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
        at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
        at org.apache.cxf.io.CachedOutputStream.write(CachedOutputStream.java:461)
        at org.apache.cxf.helpers.IOUtils.copy(IOUtils.java:160)
        at org.apache.cxf.helpers.IOUtils.copy(IOUtils.java:104)
        at org.apache.cxf.attachment.AttachmentDataSource.cache(AttachmentDataSource.java:52)
        at org.apache.cxf.attachment.AttachmentDeserializer.cacheStreamedAttachments(AttachmentDeserializer.java:20
8)

If you are using CxfSerlvet, it lets you define the buffer memory (attachment-memory-threshold) and the max upload size (attachment-max-size)



If you are sure you have these set correctly, make sure the attachment-directory exists. If it does, make sure tomcat has permission to write to it.

Setting up a Java Tomcat7 Production Server on Amazon EC2

This tutorial will demonstrate how to build a Tomcat7 server running a Java application on Amazon EC2.

Here are the tools we will set up:

• Apache Tomcat7
• Open JDK7
• GitHub
• Maven 3.0.4
• MySQL

Creating a EC2 Instance

In the AWS Management Console, begin by creating a t1.micro Ubuntu Server 12.04.1 LTS 64-bit machine. (If you don't know how to create an instance, read Amazon EC2 - Launching Ubuntu SErver 12.04.1 LTS step by step guide.)

Here are some guidelines:

• Uncheck Delete on Termination for the root volume
• Add port 22, 80, 443 to the Security Group.

Create a EBS volume

We will create a 20GB volume to store our Java code. The EBS will be formated with XFS.

Read Amazon EC2 - Mounting a EBS drive.

If the volume kept on getting stuck, keep restarting the EC2 instance until it's attached.

Configure the EC2 instance

ssh into the instance (ssh -i {key} ubuntu@{your_ec2_public_address})

sudo apt-get update -y

My mounting point for /dev/xvdf is called /vol.

cd /vol

mkdir src

mkdir webapps

mkdir war_backups

/vol/src is where we will place the application code. /vol/webapps is where we will deploy the WAR file. /vol/war_backups is for making war backups, as the name implies.

Deploying code from GitHub

Skip this if you are using other source control. The idea is that we will put the Java application code in the /vol/src folder.

sudo apt-get install git -y
mkdir /vol/src
cd /vol/src

git config --global user.name "your_name"
git config --global user.email "your_email"
git config --global github.user "your_github_login"
git clone ssh://git@github.com/username/repo.git

You will want to establish a connection with Github using ssh rather than https because if you are building an image that can be used for auto-scaling you don't want to input the username and password every time. See Generating SSH Keys for more details.

Your project should be located in /vol/src/{your_project}


Set up the Tomcat7 server

Begin by reading Install OpenJDK 7. Read Install Java OpenJDK 7 on Amazon EC2 Ubuntu.

echo $JAVA_HOME to check if it's set.

Install Tomcat 7. Read Install Tomcat 7 on Amazon EC2 Ubuntu.

Remember to change to ports 80, 443, and the root web directory as the Tomcat war root path.

Check http://{your_ec2_public_address} in your browser to make sure Tomcat7 is running.

Make sure Tomcat7 is still up after you reboot the machine.


Generating the war file

We will be using Maven to compile our Spring Java project. If you are using other build frameworks, skip this.

Read Install Maven 3 on Amazon EC2 Ubuntu.

Run "mvn --version" to make sure it's using OpenJDK 7 and running the latest version of Maven.

cd /vol/src/{your_project}

mvn clean install

A WAR file should be built.

Move this WAR file into the Tomcat webapps directory. If you are following this tutorial, it should be at /vol/webapps.

Remember to label this WAR file as ROOT.war

It's easier to do load balancing mapping later.

Browse to check you can access the site.

Using Amazon SES as the SMTP email service

Using SES will increase the likelihood of email delivery. Read Using Amazon SES to send emails.

Recompile your project and test it.

Moving MySQL to Amazon RDS

If you are using MySQL, you should move to Amazon RDS as it simplifies a lot of management, backup operations for you.

Read Using MySQL on Amazon RDS.

To interact with RDS through your EC2 instance, install MySQL Server or just the MySQL client interface.

sudo apt-get install -y mysql-server

Stop the local MySQL server. We won't be using it.

sudo /etc/init.d/mysql stop

Connect to your RDS instance

mysql -h {rds_public_address} -P 3306 -u{username} -p{password}

Do NOT use the following form. You will get a access denied.

mysql -u{username} -p{password} -h {rds_public_address} -P 3306

Update the JDBC settings in your application, recompile and test it.


Load Balancing Tomcat7

If you are planning to run multiple instances, read Setting up Lighttpd Load Balancer on EC2 Ubuntu or Setting up Nginx on EC2 Ubuntu.

Install Tomcat 7 on Amazon EC2 Ubuntu

Install tomcat7

sudo apt-get install tomcat7

Start tomcat7

sudo service tomcat7 start

Stop tomcat7

sudo service tomcat7 stop

Restart tomcat7

sudo service tomcat7 restart

Tomcat7 runs on port 8080 by default. Remember to add port 8080 to the security group in the Amazon EC2 Management Console.


Change from port 8080/8443 to 80/443

sudo vi /etc/tomcat7/server.xml

Enable AUTHBIND

sudo vi /etc/default/tomcat7 

# If you run Tomcat on port numbers that are all higher than 1023, then you
# do not need authbind.  It is used for binding Tomcat to lower port numbers.
# NOTE: authbind works only with IPv4.  Do not enable it when using IPv6.
# (yes/no, default: no)
#AUTHBIND=no 

AUTHBIND=yes

Create AUTOBIND permission files

touch /etc/authbind/byport/80
touch /etc/authbind/byport/443
chmod 0755 /etc/authbind/byport/80
chmod 0755 /etc/authbind/byport/443
chown tomcat7:tomcat7 /etc/authbind/byport/80
chown tomcat7:tomcat7 /etc/authbind/byport/443

Restart Tomcat7

sudo service tomcat7 restart

Check if Tomcat is running.

netstat -tulpn

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:8005          0.0.0.0:*               LISTEN      1268/java    
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      18848/mysqld  
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1268/java    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      619/sshd      
tcp6       0      0 :::22                   :::*                    LISTEN      619/sshd      
udp        0      0 0.0.0.0:68              0.0.0.0:*                           431/dhclient3  

Tomcat7 is running on port 80 as process "java".

Deployment root path: /var/lib/tomcat7/webapps
Log path: /var/log/tomcat7

You can change the deployment root path in /etc/tomcat7/server.xml. Search for "webapps".

You will also need to change logging.properties

Search for

1catalina.org.apache.juli.FileHandler.level = FINE
1catalina.org.apache.juli.FileHandler.directory = /vol1/logs
1catalina.org.apache.juli.FileHandler.prefix = catalina.
2localhost.org.apache.juli.FileHandler.level = FINE
2localhost.org.apache.juli.FileHandler.directory = /vol1/logs
2localhost.org.apache.juli.FileHandler.prefix = localhost.

CATALINA_HOME and CATALINA_BASE are defined in:

vi /etc/init.d/tomcat7 

# Directory where the Tomcat 6 binary distribution resides
CATALINA_HOME=/usr/share/$NAME
# Directory for per-instance configuration files and webapps
CATALINA_BASE=/var/lib/$NAME

$NAME is tomcat7


Change webapps location to EBS volume:

sudo vi /etc/tomcat7/server.xml

Find

Change appBase to "/vol/webapps" or your desired location.

/vol is the mounted point of a EBS volume. To create a EBS volume, read the post - Amazon EC2 - Mounting a EBS drive.

Restart tomcat7.

Deploy a war file to "/vol/webapps" and access the application at

http://{ec2-domain}/{app_name}