Study Flashcards For Android

During the past month, I worked on a new version of StudyFlashcard For Android.

StudyFlashcard 3.0 is a mobile flashcards application for students to learn foreign languages, review exam materials, or simply as a memorization tool.

Here is the UI of the application:

StudyFlashcard Android is different from other flashcard applications in the following ways:

1) Organize flashcards with #hashtags

Powerful #hashtag system lets you add flashcards to your midterms, exams, favorite lists, or secret lists. Study in whatever ways you imagine.

2) Study multiple subjects at once with Merge Play

Whether you have three exams on the same day or you are just bored of studying one subject for too long, Merge Play lets you study multiple flashcard sets together.

3) Study multiple questions at once with Cram Mode (Web only)

Cram mode offers a multi-column view that lets you study a lot of questions at once. This is perfect for a quick memory refresh before exams.

4) Perfect for learning vocabularies and foreign languages with three sided flashcards

Each StudyFlashcard has three sides: question, answer, and hint. Three-sided flashcards are perfect for learning Asian languages such as Chinese and Japanese.

Full list of features:

• Sync Flashcards with StudyFlashcard.com

• Score tracking

• hashtags

• play multiple sets of flashcards/quizzes at the same time

• Sign up is not necessary

• Works completely offline (You can create flashcards offline)

• Three sided flashcards/quizzes with question, answer and hint

• Can adjust text size

Give StudyFlashcard Android a try. Let us know what you think.

JMeter Tutorial - Writing a Test Plan

In this tutorial, we will write a very simple test plan.

Here are the specs of the Test Application we will run JMeter against:

• large instance on EC2
• Ubuntu LTS12.04
• Tomcat 7
• Java and Spring

Setting up JMeter on Windows

We will be setting up JMeter on Windows simply because it's easier. Read User Load Testing Simulation - Installing Apache JMeter on Windows 2012 Base.


Creating a Thread Group

Start JMeter by clicking on bin/ApacheJMeter.jar in your install location. You will see in the Tree hierarchy that it has a Test Plan item and a WorkBench item.

Right click on Test Plan -> Add -> Threads (Users) -> Thread Group

The Thread Group tells JMeter how many users and requests it should simulate.

In the Thread Group panel, fill in the following:

Name: Web Users
Number of Threads (users): 10
Ramp-Up Period (in seconds): 0
Loop Count: 5

The above will generate 5 requests for each of the 10 users. Total number of requests is 50.

The Ramp-Up Period defines the time delay which each JMeter will start the user. If Ramp-Up Period is 10 and Number of Threads is 10 then the delay between each user is 1 second. In the above, we have Ramp-Up Period = 0, meaning that all the users will start at the same time.


Add Default HTTP Request Information

Since most requests we are going to make will share some common properties (like IP and port), we will set up some default HTTP Request Information.

Right click on Web Users -> Add -> Config Element -> HTTP Request Defaults.

Fill in your server name (or IP) and port number.


Add Cookies

We will add HTTP Cookies.

Right click on Web Users -> Add -> Config Element -> HTTP Cookie Manager


Adding a HTTP Request

This is where your simulated users will request. We will add a login request. You will need to figure out what are your site's username and password parameters (In my case, it's username and password)

Right click on Web Users -> Add -> Sampler -> HTTP Request

Fill in the following:

Name: Login
Server Name of IP: /
Method: POST

In parameters, click Add.

Add Name=username, Value={your_username}
Add Name=password, Value={your password}

You can add a GET a request similarly as well.


Adding the Graph Results Listener

To observe response time, we can add the Graph Results Listener.

Right click on Web Users -> Add -> Listener -> Graph Results.


Running the Test Plan

In the menu bar, click on Run -> Start

CxfSerlvet OutOfMemoryError

I was working with uploading large files to my Spring application, and I encountered an OutOfMemoryError.

java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOf(Arrays.java:2271)
        at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
        at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
        at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
        at org.apache.cxf.io.CachedOutputStream.write(CachedOutputStream.java:461)
        at org.apache.cxf.helpers.IOUtils.copy(IOUtils.java:160)
        at org.apache.cxf.helpers.IOUtils.copy(IOUtils.java:104)
        at org.apache.cxf.attachment.AttachmentDataSource.cache(AttachmentDataSource.java:52)
        at org.apache.cxf.attachment.AttachmentDeserializer.cacheStreamedAttachments(AttachmentDeserializer.java:20
8)

If you are using CxfSerlvet, it lets you define the buffer memory (attachment-memory-threshold) and the max upload size (attachment-max-size)



If you are sure you have these set correctly, make sure the attachment-directory exists. If it does, make sure tomcat has permission to write to it.

Munin not generating graphs - Make sure CRON job is running

I am currently using Ubuntu 12.04 on EC2.

If your munin master is not running, you should check if munin is set up as a CRON job.

List all the scheduled cron jobs:

crontab -l

If munin-cron is not set up, we will add it. Edit the crontab file

crontab -e

Let's make munin master run every 5 mins. Append the following to the end of the file

*/5 * * * * /usr/bin/munin-cron

Let's make munin run.

sudo -u munin munin-cron

Enable Async Request Processing for Java Spring

Serlvet 3 supports asynchronous request processing which allows the requested operation to be performed on a separate thread, freeing the HTTP request memory.

In Spring, an allocated amount of memory is allocated for request processing. If you are running a long operation (such as uploading a large file, batch mailing, big data analysis), the memory for that request would be blocked until the operation is finished.

A better way of approaching this would be to return the request right away and let the long operation to run in another thread.

In Spring 3.x, it is relatively easy with the @Async support.

Before I dive down into the tutorial, here's my specs:

• ubuntu 12.04
• large EC2 instance
• OpenJDK 1.7
• Maven 3.0.4
• Spring 3.2.4
• CXFServlet 2.5.3 with JAX-RS (This is for Rest API)

This post will be about how to enable @Async in your spring project.

Install the latest version of Spring

At the time of this writing, the stable version is 3.2.4. The following shows how to add the maven dependency in pom.xml

Use Serlvet 3.0 namespace

In web.xml, make sure you use the XML nameplace for version 3.0.

 

Add Async Support in entry servlets in web.xml

If you are writing a normal Spring MVC Web app, you will need to add async-supported in the dispatcher serlvet below. My project only uses the Rest API, so I put the async-supported xml tag in the CXFServlet.

Enable Async in application context

In applicationContext.xml, enable executor and scheduler. If you don't know where is the applicationContext.xml, find it from web.xml's contextConfigLocation context-param field.

Make sure you have the task namespace as above.

Annotate a method as @Async

Below is a quick test of the @Async method. Notice the @Async annotation below.

Try taking out the @Async annotation and execute the program. You should be able to observe the difference.

Elastic Search on EC2 - Install ES cluster on Amazon Linux AMI

We will install ElasticSearch (ES) on a EC2 instance.

Here's the specs:

• Amazon Linux AMI 2013.09
• Medium instance
• 64-bit machine
• Elastic Search 0.90.5
• Spring MVC
• Maven

Begin by launching an instance.  You may get an out of memory error in /var/log/syslog if you use a micro instance when you launch a machine.  If you are not sure how to launch an instance, read Amazon EC2 - Launching Ubuntu Server 12.04.1 LTS step by step guide.

For the security group, you will need to open the following ports:

• 22 (SSH)
• 9300 (ElasticSearch Transport)
• 9200 (HTTP Testing)

Attach Two EBS drives

We will be using one for saving data and one for logging.  Create and attach two EBS drives in the AWS console.

You will have two volumes: /dev/xvdf and /dev/xvdg.  Let's format them using XFS.

yum -y install xfsprogs xfsdump
sudo mkfs.xfs /dev/xvdf
sudo mkfs.xfs /dev/xvdg

Make the data drive /vol. Make the log drive /vol1.

vi /etc/fstab

Append the following:

/dev/xvdf /vol xfs noatime 0 0
/dev/xvdg /vo1 xfs noatime 0 0

Mount the drives

mkdir /vol
mkdir /vol1
mount /vol
mount /vol1

Read Amazon EC2 - Mounting a EBS drive for more information.

ssh into the instance

ssh -i {key} ubuntu@{ec2_public_address}

Update the machine

sudo yum -y update

Install Oracle Sun Java

In order to run ES efficiently, a JVM must be able to allocate large virtual address space and perform garbage collection on large heaps without pausing JVM.  There are also some stories online talking about OpenJDK is not as good as Oracle Java for ES.  Feel free to let me know in the comments below if this is not the case.

Download Java 7 from Oracle.

Put it in /usr/lib/jvm.

Extract and install it

tar -zxvf jdk-7u40-linux-x64.gz

Rename the folder from jdk1.7.0_40 to jdk1.7.0

You should now have jdk1.7.0 inside /usr/lib/jvm

Set java, javac.

sudo /usr/sbin/alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0/bin/java" 1
sudo /usr/sbin/alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0/bin/javac" 1

Correct the permissions.

sudo chmod a+x /usr/bin/java
sudo chmod a+x /usr/bin/javac
sudo chown -R root:root /usr/lib/jvm/jdk1.7.0

Set to the Sun Java by:

sudo /usr/sbin/alternatives --config java

Check your java version.

java -version

Download and install ElasticSearch

Download ElasticSearch (Current version as of this writing is 0.90.5).

sudo su
mkdir /opt/tools
cd /opt/tools
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.5.zip
unzip elasticsearch-0.90.5.zip

Install ElasticSearch Cloud AWS plugin.

cd elasticsearch-0.90.5
bin/plugin -install elasticsearch/elasticsearch-cloud-aws/1.15.0

Configuring ES

AWS can shut down your instances at any time.  If you are storing indexed data in ephemeral drives, you will lose all the data when all the instances are shut down.

There are were two ways to persist data:

• Store data in EBS via local gateway
• Store data in S3 via S3 gateway

A restart of the nodes would begin to recover data from the gateway. The EBS route is better for performance, while the S3 route is better for persistence [S3 is deprecated].

We will be setting up a ES cluster and use a local gateway. S3 gateway is deprecated at the time of this writing.  The ES team has promised a new backup mechanism in the future.

vi /opt/tools/elasticsearch-0.90.5/config/elasticsearch.yml

cluster.name: mycluster

cloud:

    aws:

        access_key:

        secret_key:
        region: us-east-1

discovery:

    type: ec2

We have specified a cluster called "mycluster" above. You will need to input your aws access keys and create a S3 bucket.

We also need to ensure the JVM does not swap by doing two things:

1) Locking the memory (find this setting inside elasticsearch.yml)

bootstrap.mlockall: true

2) Set ES_MIN_MEM and ES_MAX_MEM to the same value. It is also recommended to set them to half of the system's available ram. We will set this in the ElasticSearch Service Wrapper later in the article.

Create the data and log paths.

mkdir /vol/elasticsearch/data
mkdir /vol1/elasticsearch/log

Set the data and log paths in /config/elasticsearch.yml

path.data: /vol/elasticsearch/data
path.logs: /vol1/elasticsearch/logs 

Let's edit config/logging.yml

vi /opt/tools/elasticsearch-0.90.5/config/logging.yml

Edit these settings and make sure these lines are uncommented and present

logger:

  gateway: DEBUG

  org.apache: WARN

  discovery: TRACE

Testing the cluster

bin/elasticsearch -f

Browse to the ec2 address at port 9200

http://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:9200/

You should see the following:

{
  "ok" : true,
  "status" : 200,
  "name" : "Storm",
  "version" : {
    "number" : "0.90.5",
    "build_hash" : "c8714e8e0620b62638f660f6144831792b9dedee",
    "build_timestamp" : "2013-09-17T12:50:20Z",
    "build_snapshot" : false,
    "lucene_version" : "4.4"
  },
  "tagline" : "You Know, for Search" 

}


Installing ElasticSearch as a Service

We will be using the ElasticSearch Java Service Wrapper.

Download the service wrapper and move it to bin/service.

curl -L -k http://github.com/elasticsearch/elasticsearch-servicewrapper/tarball/master | tar -xz
mv /service /opt/tools/elasticsearch-0.90.5/bin

Make ElasticSearch to start automatically when system reboots.

bin/service/elasticsearch install

Make ElasticSearch Service a defaul command (we will call this es_service)

ln -s /opt/tools/elasticsearch-0.90.5/bin/service/elasticsearch /usr/bin/es_service

Start the service

es_service start

You should see:

Starting ElasticSearch...
Waiting for ElasticSearch......
running: PID:2503 

Tweaking the memory settings

There will be three settings you want to care about:

• ES_HEAP_SIZE
• ES_MIN_MEM
• ES_MAX_MEM

It is recommended to set ES_MIN_MEM to be the same as ES_MAX_MEM.  However, you can just set ES_HEAP_SIZE as it will be assigned to both ES_MIN_MEM and ES_MAX_MEM.

We will be tweaking these settings in the service wrapper's elasticsearch.conf instead of elasticsearch's.

vi /opt/tools/elasticsearch-0.90.5/bin/service/elasticsearch.conf

set.default.ES_HEAP_SIZE=1024

There are a few things you need to beware of.

1. You need to leave some memory for the OS for non elasticsearch operations. Try leaving at least half of the available memory.
2. As a reference, use 1024Mb for every 1 million documents you are saving.

Restart the service.

Ubuntu EC2 - Install Sun Oracle Java

Download Java 7 from Oracle.

Put it in /usr/lib/jvm.

Extract and install it

tar -zxvf jdk-7u40-linux-x64.gz

Rename the folder from jdk1.7.0_40 to jdk1.7.0

You should now have jdk1.7.0 inside /usr/lib/jvm

Set java, javac.

sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0/bin/java" 1
sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0/bin/javac" 1

Correct the permissions.

sudo chmod a+x /usr/bin/java
sudo chmod a+x /usr/bin/javac
sudo chown -R root:root /usr/lib/jvm/jdk1.7.0

If you have more than one version of java, you can always switch them using

sudo update-alternatives --config java

Check your java version.

java -version