Deploying Rails Application on Windows Azure (Ubuntu VM)

In the last weeks, I've been exploring my options for hosting a Ruby on Rails application. An important step was Windows Azure. I registered for a 90-day trial to give it a try. One of the great services Microsoft has offered was the Ubuntu virtual machine ( :D ). So this tutorial can be useful even with a normal Ubuntu machine (except the Endpoints part) since I'm treating it like an Ubuntu machine regardless of the Azure service it is hosted on.

In this tutorial, I will create a new Ubuntu 12.04 LTS virtual machine, install Ruby ( 1.9.3 ) and Rails ( 3.2.9), install Passenger gem, install Apache server and connect it to Rails application. And finally, some important notes about getting your project up and running.

Create an Ubuntu 12.04 LTS VM:

- In you portal, from the bottom right corner click: New -> Compute -> Virtual Machine -> From Gallery.

- Scroll the list to Ubuntu Server 12.04 LTS and click it.

- Fill the required data about the VM name, the username to create in that machine and password (you will need them A LOT), and number of cores running this machine (I'm on trial version, so 1 is enough to try it).

- Choose the appropriate DNS name (your service URL) and the suitable place for the service.

- You are done with the VM creation.

Create Endpoints for SSH, FTP, Apache:

- The endpoint is the way of communication between the VM and the outer world. Each endpoint takes a public port (the one you call) and a private port (the one the VM listens to).

- Click on the created VM, select the ENDPOINTS tab. You will find that the SSH endpoint is already created on port 22.

- Now create two other ports for FTP (port 21) and Apache service (default port, port 80). Names of the endpoints do not matter, but you'd better give them meaningful names.

- Do not forget to start the VM before the next step.

Access the Ubuntu Server Through SSH:

- All you interaction with the server will be though SSH via port 22. Although you can then install a GUI package and remote access the server, I choose not to do this because of the extra space and CPU cycles it takes, and common, this is Linux, you only need a terminal to have fun!

- Since I'm using windows, I use a nice SSH client: PuTTY. Just download, run, enter your server's name and click 'open', enter usrename/password in the terminal that shows up.

- Now you are on board, let's install Ruby and Rails.

Note: You can copy and paste inside PuTTY terminal as follows: highlight the terminal text with the mouse to copy it, and right click on the terminal to paste.

Install Ruby and Rails:

- Some people prefer to install Ruby using 'rbenv' or 'rvm', but I prefer simplicity. I'll just install Ruby directly.

- If you cannot 'sudo apt-get install ruby -v 1.9.3' directly, you may follow these steps:

sudo apt-get update

sudo apt-get install ruby1.9.1 ruby1.9.1-dev \
  rubygems1.9.1 irb1.9.1 ri1.9.1 rdoc1.9.1 \
  build-essential libopenssl-ruby1.9.1 libssl-dev zlib1g-dev

sudo update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.9.1 400 \
         --slave   /usr/share/man/man1/ruby.1.gz ruby.1.gz \
                        /usr/share/man/man1/ruby1.9.1.1.gz \
        --slave   /usr/bin/ri ri /usr/bin/ri1.9.1 \
        --slave   /usr/bin/irb irb /usr/bin/irb1.9.1 \
        --slave   /usr/bin/rdoc rdoc /usr/bin/rdoc1.9.1

# choose your interpreter
# changes symlinks for /usr/bin/ruby , /usr/bin/gem
# /usr/bin/irb, /usr/bin/ri and man (1) ruby
sudo update-alternatives --config ruby
sudo update-alternatives --config gem

# now try
ruby --version

Then install Rails

sudo gem install rails --no-rdoc --no-ri

Install Passenger and Apache:

- The passenger gem is the connection between Apache server and your rails app. We need to install Passenger, Apache, and the Passenger's Apache module.
(edit: I've been facing some issues with v3 when restarting the server, so I added --pre to install the yet-not-released v4. In case you have v4 by the time you read this, just proceed with the command as it is.)

sudo gem install passenger --no-rdoc --no-ri
sudo passenger-install-apache2-module

- The previous line will open an installation wizard to guide you though the installation of the module. It is supposed to ask you for extra packages to install. In my case, these were the packages:

sudo apt-get install libcurl4-openssl-dev apache2-mpm-prefork apache2-prefork-dev libapr1-dev libaprutil1-dev

- After installing the missing packages, run the module installation one more time.

sudo passenger-install-apache2-module

- Now the module will install and ask you to add some paths to the apache config file which is supposed to be in 'etc/apache2/apache2.conf'.

- It will also give you an example of how to add a virtual host to get your rails app running through Apache.

- To open the file from terminal, use:

sudo nano etc/apache2/apache2.conf

- Here is what I had to add to my config file:

LoadModule passenger_module /var/lib/gems/1.9.1/gems/passenger-3.0.18/ext/apache2/mod_passenger.so
PassengerRoot /var/lib/gems/1.9.1/gems/passenger-3.0.18
PassengerRuby /usr/bin/ruby1.9.1

<VirtualHost *:80>
  ServerName adly-test.cloudapp.net
  # !!! Be sure to point DocumentRoot to 'public'!
  DocumentRoot /home/adly/apps/testrails/public

  <Directory /home/adly/apps/testrails/public>
     # This relaxes Apache security settings.
     AllowOverride all
     # MultiViews must be turned off.
     Options -MultiViews
  </Directory>
</VirtualHost>

Upload Your Project Using FTP:

- Some tutorials go for using Github and some packaging tools, but I do not like that. I want to upload my files myself. So I'm using "vsftpd" for the Ubuntu Server side, and FileZilla on the my Windows side.
(You can use Git without the need to Capistrano or Unicorn gems, remember it is just a Linux machine. Git is better for continuous development.) (you can also use WinSCP client for SFTP protocol and forget about FTP client and end point)

- Install vsftpd

sudo apt-get install vsftpd

- Edit the config file:

sudo nano /etc/vsftpd.conf

- Modify the following lines:

#local_enable=YES

to

local_enable=YES

#write_enable=YES

to

write_enable=YES

anonymous_enable=YES

to

anonymous_enable=NO

- Restart the FTP server

sudo service vsftpd restart

- Now we are done with the server side. On the client side, install FileZilla and open it.

- From "Edit" -> "Settings", make sure the settings are as follows:

- From the main screen, enter your service URL, username, password, and port 21 and click "Quick Connect".

- Now you will have the server folders on the right, and your local files on the left. Move and modify however you want.

Fine tuning to get things working right (do not trust HelloWorld tutorials):

If you create a dummy project using 'rails new testrails', it will probably work fine. But those tutorials do not give the complete case of a real application, so here are some extra steps to do to avoid errors or unknown behavior.

Rails and Apache log file:

- It happens that the Rails app will write to the Apache log file, so it is better to locate a custom path to that log file instead of the default path. I prefer logging errors in a log file in the same directory of Rails log folder. So I add the following line to the VirtualHost configuration in the apache config file. I also prefer declaring the Rails working environment explicitly to avoid any human/machine errors in the future.

ErrorLog /home/adly/apps/testrails/log/error.log
RailsEnv production

so it is now like

LoadModule passenger_module /var/lib/gems/1.9.1/gems/passenger-3.0.18/ext/apache2/mod_passenger.so
PassengerRoot /var/lib/gems/1.9.1/gems/passenger-3.0.18
PassengerRuby /usr/bin/ruby1.9.1

<VirtualHost *:80>
  ServerName adly-test.cloudapp.net
  # !!! Be sure to point DocumentRoot to 'public'!
  DocumentRoot /home/adly/apps/testrails/public
  ErrorLog /home/adly/apps/testrails/log/error.log
  RailsEnv production

  <Directory /home/adly/apps/testrails/public>
     # This relaxes Apache security settings.
     AllowOverride all
     # MultiViews must be turned off.
     Options -MultiViews
  </Directory>
</VirtualHost>

Assets Compilation:

- You have to set this line to true

config.assets.compile = true

in {app_root}/config/environments/production.rb

and do not forget to compile the assets whenever changed:

rake assets:precompile

[check the "ExecJS Error, and libv8 / therubyracer version errors" section at the bottom if you have an error]

DB migration:

- If you have a database in your project, the declaration of "production" environment in the apache config file is not enough, you have to declare it explicitly when migrating:

rake db:migrate RAILS_ENV="production"

FILES PERMISSIONS:

- This is a very important step, since some files are still owned by you with zero permissions to other users.

- The quick and not-so-liberal permission is 755 for all the application folder

sudo chmod -R 755 /path/to/your/app/

- (just in case you face problems later, make this step) Double check that you give rwx privileges to the DB, assets, log and tmp folders for the user running this app whenever you add new files. For example:

sudo chmod -R 755 /path/to/your/app/app/assets
sudo chmod -R 755 /path/to/your/app/db
sudo chmod -R 755 /path/to/your/app/tmp
sudo chmod -R 755 /path/to/your/app/log

- Note: Do not forget to check the permissions of any new file you upload.


ExecJS Error, and libv8 / therubyracer version errors:

- Now this is something very common with a lot of workarounds. It is on Rails 3 because of some compatibility issues. You should get an error like "Could not find a JavaScript runtime. See https://github.com/sstephenson/execjs for a list of available runtimes.".

- To solve this problem, open your Gemfile, and add the following lines under the "group :assets do" section:

gem 'therubyracer', '0.11.0beta8', :platforms => :ruby
gem 'libv8', '~> 3.11.8.3', :platform => :ruby

then do not forget to "bundle install". No need for 'sudo', let the gems be installed locally.

[update: libv8 compilation may take several minutes if there is no matching binary for your OS version and specified version number. On another server I had to change the version to '3.11.8.3' to save time and CPU usage. Click here for more: Installing with native extensions stall ]

One final note: Do not forget to restart Apache server for the changes to take effects.

sudo service apache2 restart

Ruby on Rails Facebook Application using Koala Gem

Here is a another Facebook application example (check the python example), but this time it is with Ruby on Rails using Koala gem. The great thing about Koala is how it is easy integrated and straight forward. Here are the steps from the very beginning:

1- Create a new Ruby on Rails project:

rails new rorApp

2- Delete ./public/index.html file as it is not needed.


3- In Gemfile add:

gem 'koala', '1.3.0'

4- In ./config/initializers folder, add a constants.rb file with the following data:

APP_ID= '123456789' # please change!
APP_SECRET= '1b2n3n5n6n7m8m9n9m0m' # please change!
SITE_URL = 'http://localhost:3000/' # please change!

Where APP_ID and APP_SECRET are the values you have from the Facebook application you create in the developers section, and the SITE_URL is the root URL of your application website ('http://localhost:3000/' if our case of testing locally)


5- In ./config/routes.rb, add the following routes:

root :to => 'home#index'

match '/index' => 'home#index'
match '/login' => 'home#login'

The first line makes calling the root of your application points to your index page.

6- In ./app/views folder, create a 'home' folder and inside it create a 'index.html.erb' with any content you want to show. It will only be available after user logs in.

7- In ./app/controller folder, create a 'home_controller.rb' file with the following content:

class HomeController < ApplicationController
            
    def index   
        if params[:code]
            # acknowledge code and get access token from FB
            session[:access_token] = session[:oauth].get_access_token(params[:code])
        end  

        # auth established, now do a graph call:
        @api = Koala::Facebook::API.new(session[:access_token])

        begin
            @user_profile = @api.get_object("me")
        rescue Exception=>ex
            puts ex.message
            #if user is not logged in and an exception is caught, redirect to the page where logging in is requested
            redirect_to '/login' and return
        end

        respond_to do |format|
         format.html {   }    
        end
    end
    
    #########################################################
    
    def login
        session[:oauth] = Koala::Facebook::OAuth.new(APP_ID, APP_SECRET, SITE_URL + '/')
        @auth_url =  session[:oauth].url_for_oauth_code(:permissions=>"read_stream publish_stream")  

        redirect_to @auth_url
    end
    
    #########################################################
end

The code explains itself, but here is a quick explanation: When the index page is called, the application looks for the access token for the current user in the cookies ('session[]'). If not found, it will redirect to the login page. The login handler will do the authentication headache for you with the extra permissions you want and passes the root URL as a call back so that Facebook will call it after authentication is done. Then when the index page is called one more time with an authentication code, it will be used to get the access token needed for any later requests.
So when the user opens the index page, he will notice some redirects and message boxes asking for allowing your application and for extra permissions then he is redirected to your index page.

8- Now you have your application ready. Just run 'bundle' from your terminal (make sure you are in the root folder of your application) to install Koala and then 'rails s' to start the web service.

bundle
rails s

9- Finally here are some Graph API calls that you may use to get profile data, friends list, post text/image and text to user's wall.

@api = Koala::Facebook::API.new(session[:access_token])
@user_profile = @api.get_object("me")
@friends = @api.get_connections(user_profile["id"], "friends")
@api.put_wall_post("hi")
@api.put_picture("http://www.example.com/image.png", {:message => "an image of mine!"})

You can learn more about Koala gem from this link: https://github.com/arsduo/koala

Google App Engine Facebook Application Example Using Python

This is a detailed example built on Facebook Python SDK example. In this example I'll take it step by step from the very beginning to login, get user data, get friends list, invite friends, and post to wall. So let's start. (also check the Ruby on Rails with Koala example)

- Download "facebook.py" from this link on GitHub.

- Add the downloaded file to your Google App Engine project.

- You can now make you own Facebook application code or re-use the current example (which I choose to do).

- Take these global variables and imports into your application (your GAE python file)

FACEBOOK_APP_ID = "123456789" #your own FB app id here
FACEBOOK_APP_SECRET = "756483568435475" #your own FB app secret here
INVITATION_TEXT = "I invite you to try my app. It is amazing!"

import facebook
import os.path
import wsgiref.handlers
import logging
import urllib2
import hashlib

from google.appengine.ext import db
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
from google.appengine.ext.webapp import template
from google.appengine.api.urlfetch import fetch

import webapp2

- Add this Model and Base Class that will be used to determine the user ID from cookies

class User(db.Model):
    id = db.StringProperty(required=True)
    created = db.DateTimeProperty(auto_now_add=True)
    updated = db.DateTimeProperty(auto_now=True)
    name = db.StringProperty(required=True)
    profile_url = db.StringProperty(required=True)
    access_token = db.StringProperty(required=True)

class BaseHandler(webapp.RequestHandler):
    """Provides access to the active Facebook user in self.current_user

    The property is lazy-loaded on first access, using the cookie saved
    by the Facebook JavaScript SDK to determine the user ID of the active
    user. See http://developers.facebook.com/docs/authentication/ for
    more information.
    """
    @property
    def current_user(self):
        if not hasattr(self, "_current_user"):
            self._current_user = None
            cookie = facebook.get_user_from_cookie(
                self.request.cookies, FACEBOOK_APP_ID, FACEBOOK_APP_SECRET)
            if cookie:
                # Store a local instance of the user data so we don't need
                # a round-trip to Facebook on every request
                user = User.get_by_key_name(cookie["uid"])
                if not user:
                    graph = facebook.GraphAPI(cookie["access_token"])
                    profile = graph.get_object("me")
                    user = User(key_name=str(profile["id"]),
                                id=str(profile["id"]),
                                name=profile["name"],
                                profile_url=profile["link"],
                                access_token=cookie["access_token"])
                    user.put()
                elif user.access_token != cookie["access_token"]:
                    user.access_token = cookie["access_token"]
                    user.put()
                self._current_user = user
        return self._current_user

- Now let's create our handler. Note that in case you wan to use your application inside Facebook frame, you will have to handle POST request too.

- Logging In and Inviting Friends

class HomeHandler(BaseHandler):
    def get(self):
        self.show_main()

    def post(self):
        self.show_main()
        
    def show_main(self):
        path = os.path.join(os.path.dirname(__file__), "templates/main.html")
        args = dict(current_user=self.current_user,
                    facebook_app_id=FACEBOOK_APP_ID,
                    invitation_text=INVITATION_TEXT)
        self.response.out.write(template.render(path, args))

- Now let's create the template for the HTML template. Here is how it goes: If the passed "current_user" parameter is valid, it will show the content of your app and a link to a JQuery function to invite friends using Facebook Javascript SDK initialized at the bottom. Otherwise, it will show a Facebook login button.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
    <title>Application Title</title>
    
    <script type="text/javascript" src="/js/jquery-1.8.2.min.js"></script>
    <script type="text/javascript">
        $('#sendRequest').click(function() {
          FB.ui(
            {
              method  : 'apprequests',
              message : $(this).attr('data-message')
            },
            function (response) {
              // If response is null the user canceled the dialog
              if (response != null) {
                //logResponse(response);
              }
            }
          );
        });
    });
    </script>
  </head>
  <body>
  
    {% if not current_user %}
    <div style="margin:0 auto; text-align:center;">
        <fb:login-button autologoutlink="false" scope="publish_stream"></fb:login-button>
    </div> 
    
    {% else %}
    
    <br>
    Welcome! Here is my application.
    <br>
    <a href="#" id="sendRequest" data-message="{{invitation_text}}">
      Send Requests
    </a>
    
    {% endif %}

    <div id="fb-root"></div>
    <script>
      window.fbAsyncInit = function() {
        FB.init({appId: '{{ facebook_app_id }}', status: true, cookie: true,
                 xfbml: true});
        FB.Event.subscribe('{% if current_user %}auth.logout{% else %}auth.login{% endif %}', function(response) {
          window.location.reload();
        });
      };
      (function() {
        var e = document.createElement('script');
        e.type = 'text/javascript';
        e.src = document.location.protocol + '//connect.facebook.net/en_US/all.js';
        e.async = true;
        document.getElementById('fb-root').appendChild(e);
      }());
    </script>
  </body>
</html>

- At this point, you should be having a working Python Google App Engine Facebook application. Next I'll go though some basic information that you may need to implement in your service. You may prefer XML, JSON, or a custom format. Anyways, here is the basic example that you may customize as you want and add more handling for special cases.

- User Profile

class ProfileHandler(BaseHandler):
    def get(self):
        self.show_profile()

    def post(self):
        self.show_profile()
        
    def show_profile(self):
        profile = ''
        current_user = self.current_user
        if current_user:
            graph = facebook.GraphAPI(current_user.access_token)
            profile = profile = graph.get_object("me")
        path = os.path.join(os.path.dirname(__file__), "templates/profile.html")
        args = dict(current_user=current_user,
                    facebook_app_id=FACEBOOK_APP_ID,
                    profile=profile)
        self.response.out.write(template.render(path, args))

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
    <title></title>
  </head>
  <body>
    
    {% if current_user %}
        {% if profile %}
            {{profile}}
        {% endif %}
    {% endif %}

  </body>
</html>

- Friends List

class FriendsHandler(BaseHandler):
    def get(self):
        self.show_friends()

    def post(self):
        self.show_friends()
        
    def show_friends(self):
        friends_list = []
        current_user = self.current_user
        if current_user:
            graph = facebook.GraphAPI(current_user.access_token)
            friends_list = graph.get_connections("me", "friends")['data']
        path = os.path.join(os.path.dirname(__file__), "templates/friends.html")
        args = dict(current_user=current_user,
                    facebook_app_id=FACEBOOK_APP_ID,
                    friends_list=friends_list)
        self.response.out.write(template.render(path, args))

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
    <title></title>
  </head>
  <body>
    
    {% if current_user %}
        {% if friends_list %}
            {{friends_list}}
        {% else %}
        {% endif %}
    {% else %}
    {% endif %}

  </body>
</html>

- Post Text to Wall (Arabic-friendly)

class PostTextHandler(BaseHandler):
    def post(self):
        text = self.request.get('text')
        if text:
            text = text.encode('utf-8') # for non-english text  
            current_user = self.current_user
            if current_user:
                graph = facebook.GraphAPI(current_user.access_token)
                graph.put_object("me", "feed", message=text)
                self.response.out.write('yes') # just a feedback reponse
            else:
                self.response.out.write('no') # just a feedback reponse
        else:
            self.response.out.write('no') # just a feedback reponse

- And off course do not forget to add these handlers to the suitable URLs for your service. (side note: the /?$ sign is useful to allow URL with/out the slash)

app = webapp2.WSGIApplication([
                            ('/', HomeHandler),
                            ('/profile/?$', ProfileHandler),
                            ('/friends/?$', FriendsHandler),
                            ('/posttext/?$', PostTextHandler)])

A Complete Guide to Install Ruby and Rails on Windows

As usual, whenever I find a problem with something and then another problem appears with no guide except Google and my experience (if any), I make my own guide. This time it is a guide to installing Ruby then Rails on Windows.

Step 1 - Install Ruby (Ruby 1.9.3-p327 while writing this post) through RubyInstaller:

It is as easy as any other Windows Installer. Just go to RubyInstaller download page and download it. One thing to notice while installing is to check the option to add Ruby executables to your PATH, and maybe file association if you wish. Note: Make sure the installation path does not have spaces (like: Program Files) to avoid any terminal errors later.

Step 2 - Install Development Kit (DevKit-tdm-32-4.5.2-20111229-1559-sfx.exe  while writing this post):

As their page says: "The RubyInstaller Development Kit (DevKit) is a MSYS/MinGW based toolkit than enables you to build many of the native C/C++ extensions available for Ruby." It will be needed to install some 'gems' for Rails later. The executable is no more than a 7zip archive that can be extracted to a folder of choice.

After extracting the folder,
- Open the command line (cmd) as administrator (Shift+Enter or right click and 'run as administrator') and browse to the devkit  directory.
- If we step to the normal step mentioned in the RubyInstaller Development Kit Wiki we will get an error like this: " registry.rb:172:in `find': unknown > > encoding name > > - CP720 (ArgumentError) ". So first, let's write this in the command line:
chcp 1256
- Next, let's initialize the devkit:
ruby dk.rb init
-  Make sure the generated "config.yml" file includes the right Ruby version.
- Next install devkit:
ruby dk.rb install
- You can follow the RubyInstaller Development Kit Wiki instructions to make sure it is installed properly.

Step 3 - Install Some Gems for Rails:

When installing Rails later, it will require some gems to install. So let's install them now.
- From command line, write:
gem install json -v  '1.7.5'
gem install coffee-rails -v '3.2.2'
- Wait for the download and installation to end.

Step 4 - Install Rails

- From command line, write:
gem install rails
- Wait for the download and installation to end.

Step 5 - First Rails Project

- From command line, cd to the folder of your choice or write the full path in the following line:
rails new ./projects/test_proj
- Wait for file creation to end, then cd to the project file.
- Write:
rails s
and your first Ruby on Rails service will be online on "localhost:3000"

Now you have your environment ready for Ruby on Rails development.

Packt Publishing reaches 1000 IT titles and celebrates with an open invitation

As Packt will be publishing its 1000 book within days, they have a surprise to match this event. As mentioned on their website: "Revisit Packt’s website between the 28th and 30th of September to redeem your gift, or sign up for an account with us now to receive a notification email.". That's a gift I did not expect at all.

If you don't know Packt Publishing, that's too bad. One big reason of why that is bad, is because -from my experience with some technical books from different publishers- Packt is one of those who care about  practice with less 'academic' style -which was boring during college and still-. And another thing that I did not know till a couple of days ago -when I was reading about their event- is the following:
"Packt supports many of the Open Source projects covered by its books through a project royalty donation, which has contributed over $400,000 to Open Source projects. As part of the 1000th book celebration Packt is allocating $30,000 to share between projects and authors."

I'v been depending on a couple of its books during the kick-off of my startup, and I suggest you give its practical style a taste.

Source: Packt Publishing reaches 1000 IT titles and celebrates with an open invitation

First appearance of Alkottab at Fekrety competition, AUC

Last Thursday was a special day for me that I wanted to share with the world. It was the awards ceremony for Fekrety (translated to: my idea) competition finalists and the first official appearance of Alkottab (maybe translated to: local school) company.

Alkottab is a new Egyptian startup for game development and animations with Arabic themes and content. And I'm proudly a co-founder of this Egyptian startup which aims to attract the youth in specific to games and animations that introduce them to their history and present, and invest in them to build a brighter future.

Fekrety is a national business idea competition organized by The American University in Cairo’s (AUC) School of Business – Entrepreneurship & Innovation Program (EIP), in cooperation with Intel. Although we did not win the money, it was an opportunity to announce our idea to the world of business and startups and listen to their feedback. And we had some small prizes too, if it deserves mentioning as it was not the main goal of entering this competition.

Let me introduce you to the team, and show you the idea pitching by Alkottab co-founder Eslam. The team from right to left is: Eslam, Omar and Mahmoud (me) as co-founders and developers, and Youmna and Maha as artists.

The summary of the pitch for the non-Arabic viewers is as I mentioned before about Alkottab, plus stating that the Arab countries have a bigger customer segment in game purchases than the USA and that's why foreign companies are interested in localizing their games. And because we are very interested in our idea, we decided to quit our jobs and put our savings in this company, Alkottab, and were hoping to get more fund by winning such a competition. Our first product is about the early days of our Egyptian revolution.

This was the first appearance, and not the last (if Allah wills) because we have the idea and passion, and building the experience, and trying to avoid the mistakes we noticed in different startups we witnessed or worked with.

So, this was a small report to share the news with you. And let's hope to see you very soon somewhere in a local/international event or on the market.

Full HTML5 Game Example with Crafty JS

A couple of days ago, I participated in a game jam called "Game Zanga". It was a three-day contest for game development with some conditions: in Arabic, related to a topic announced at the start of the three days (it was "Freedom"), and browser-based.

So it was a nice chance to test my knowledge and experience in game development (which is a new thing to me) and I decided to use Crafty (or maybe some call it CraftyJS). Crafty is a JavaScript game library using HTML5. One great power in Crafty is that it can use Canvas or DOM. Plus the concept of Components and Entities that allow you to develop the game in a OO style.

So, I thought I can post my code for anyone to learn from. Although it is not perfect code and it neads more refactoring and fine-tuning, it is still a good example of how a game can look like in Crafty. I almost did not need anything other than the API documentation on Crafty website (plus the tutorial if you are to start from the very beginning). So with decollapsing the code you'll immediately have a general overview of how the game code looks like, then collapsing the code can do the job to get to the details.

One important note, this game was intended to be an Arabic game, so here are some guidelines:
- The first icon in the home screen is the "Start" button.
- Once the game starts, you have two blue cannons on the left. Click any of them to use it.
- Your abjective is to let the blue soldiers pass ("unrwa" in code) and kill the green ones ("soldier" in code).
- Life points decrease if any green soldier passes.
- Your weapon is upgraded while playing.


Finally, you'll notice some faults in collision detection and a lot of other things in the code, but remember that it was made in only two days by me (a newbie) using Crafty which lacks the variety of full examples to learn from. Anyways, you'll get the idea of a full game in less than 1000 lines and then you can make your own best seller.

Here are some screenshots, followed by a download link.

Download game code here.

My HTML5 Canvas Profile Widget

Since I've not been innovating much outside work for a while now, I thought I could make any new "thing" to aggregate a few benefits I've been aiming for. The output was this HTML5 canvas, game-like, Egytian-themed profile widget. It was a chance to play with HTML5 canvas, and still stick to the design of the game loop. It was also a chance to change the look of my boring blogs.

You play the game. Drop the egg from the profile you want to visit. If the egg hits the nest, you earn the visit to the profile.

So without any more talking, the code is pretty simple:

• A game loop
• Check for user input (mouse clicks)
• Update the values of some objects
• Detect collisions of some objects and acting accordingly
• Draw objects

Notes:

• The javaScript code is hosted online and only referenced to in my HTML code. This was better to update the code once and apply it to all the places where I use it. (as a desktop/mobile developer, this is great).
• Loading an image at runtime made a lag of maybe one second when a egg is dropped, but it looked ugly so I commented the code regarding loading the egg images in runtime and used a single preloaded image instead.
• I've looked about three times for performance boosters. But still this does not mean this is the best to get from HTML5 canvas and Javascript. It is just about the amount of time I gave to this code.
• If I did not put much comments, it was because I thought the code is self-explaining.

I preferred to post the code here again for better syntax highlighting
[profile_widget_script.js]

var canvas = document.getElementById('canvas');
var context = canvas.getContext('2d');

var globalFPS               = 20;
var maxItemSpeed            = 2;
var eggSpeed                = 8;
var eggSize                 = 32;
var minItemHeight           = 200;
var iconSize                = 32;
var wingSize                = 32;
var nestX                   = 100;
var nestY                   = 330;
var nestMaxX                = 150;
var nestMinX                = 50;
var nestWidth               = 96;
var nestHeight              = 57;
var nestSpeed               = 2;
var mouseEventFlag          = false;
var mouseEventX             = 0;
var mouseEventY             = 0;
var wave1X                  = -10;
var wave1Y                  = 360;
var wave1Direction          = 1;
var wave2X                  = -25;
var wave2Y                  = 375;
var wave2Direction          = 1;
var wave3X                  = -40;
var wave3Y                  = 390;
var wave3Direction          = 1;
var waveXThreshold          = -50; // where to toggle the direction of wave
var waveSpeed               = 2;

// put the items you want to show here {icon to fly, redirecting url, egg to drop}
var profilesArray = [
            {
            icon:   "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/facebook_32x32.png",
            url:    "http://www.facebook.com/MahmoudAdly",
            egg:    "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/egg.png"
            },
            {
            icon:   "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/twitter_32x32.png",
            url:    "http://www.twitter.com/MahmoudAdly",
            egg:    "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/egg.png"
            },
            {
            icon:   "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/gplus_32x32.png",
            url:    "https://plus.google.com/103333225222170744758",
            egg:    "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/egg.png"
            },
            {
            icon:   "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/goodreads_32x32.png",
            url:    "http://www.goodreads.com/MahmoudAdly",
            egg:    "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/egg.png"
            },
            {
            icon:   "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/linkedin_32x32.png",
            url:    "http://www.linkedin.com/in/MahmoudAdly",
            egg:    "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/egg.png"
            },
            {
            icon:   "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/blogger0_32x32.png",
            url:    "http://3adly.blogspot.com/",
            egg:    "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/egg.png"
            },
            {
            icon:   "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/blogger1_32x32.png",
            url:    "http://free-3adly.blogspot.com/",
            egg:    "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/egg.png"
            }
            ];
            
var items = [];
var eggs = [];
var wingImage = new Image;
var nestImage = new Image();
var background = new Image();
var waveImage1 = new Image();
var waveImage2 = new Image();
var waveImage3 = new Image();
var eggImage = new Image(); // loading one egg to use it later, for performance reasons

wingImage.src = "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/wing_32x32.png";
nestImage.src = "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/nest.png";
background.src = "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/background.png";
waveImage1.src = "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/wave_1.png";
waveImage2.src = "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/wave_2.png";
waveImage3.src = "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/wave_3.png";
eggImage.src = "http://dl.dropbox.com/u/44163256/html5-profile-widget/assets/egg.png";

var nest = {
    image: nestImage,
    x: nestX,
    y: nestY,
    width: nestWidth,
    height: nestHeight,
    speed: nestSpeed,
    xDirection: 1,
    minX: nestMinX,
    maxX: nestMaxX
};

createItems();
setInterval(gameLoop, 1000/globalFPS);

function gameLoop(){
    updateUserInput();
    updateItems();
    updateEggs();
    updateNest();
    updateWaves();
    checkEggsCollision();
    drawBackground();
    drawWaves()
    drawItems();
    drawNest();
    drawEggs();
}

function createItems() {
    for(var i in profilesArray) {
        var profile = profilesArray[i];
        var img = new Image();
        img.src = profile.icon;
        
        items.push({
                x: 0- Math.random()*500, // so icons do not appear at once
                y: Math.random()*minItemHeight,
                speed: 1+Math.random()*maxItemSpeed,
                image: img,
                url: profile.url,
                egg: profile.egg,
                wingRotation: 0,
                wingRotationDirection: 1
        });
    }
}

function updateItems() {
    for(var i in items) {
        var item = items[i];
        item.x += item.speed;
        item.wingRotation += item.wingRotationDirection * 0.1
        if(item.wingRotation > Math.PI/4)
        {
            item.wingRotationDirection = -1;
        }
        else if(item.wingRotation < 0)
        {
            item.wingRotationDirection = 1;
        }
        
        // if item gets out of canvas scope
        if(item.x > canvas.width) {
            // recycle the item and set different values instead of removing it
            item.x = 0-Math.random()*100;
            item.y = Math.random()*minItemHeight;
            item.speed = 1+Math.random()*maxItemSpeed //between 2 and 5
            item.wingRotation = 0;
        }
    }
}

function updateEggs(){
    for(var i in eggs){
        var egg = eggs[i];
        egg.y += egg.speed;
        
        // if egg gets out of canvas scope
        if(egg.x > canvas.height) {
                // remove egg from array
                eggs.splice(i, 1);
        }
    }
}

function updateNest(){
    nest.x += nest.xDirection * nest.speed;
    
    if(nest.x > nest.maxX){
        nest.xDirection = -1;
    }
    else if(nest.x < nest.minX){
        nest.xDirection = 1;
    }
}

function updateWaves(){
    wave1X += wave1Direction*waveSpeed;
    if(wave1X > 0)
        wave1Direction = -1;
    else if(wave1X < waveXThreshold)
        wave1Direction = 1;
        
    
    wave2X += wave2Direction*waveSpeed;
    if(wave2X > 0)
        wave2Direction = -1;
    else if(wave2X < waveXThreshold)
        wave2Direction = 1;
    
    wave3X += wave3Direction*waveSpeed;
    if(wave3X > 0)
        wave3Direction = -1;
    else if(wave3X < waveXThreshold)
        wave3Direction = 1;
    
}

function checkEggsCollision(){
    for(var i in eggs){
        var egg = eggs[i];
        if(egg.x+eggSize > nest.x+nest.width/3
            && egg.x < nest.x+2*nest.width/3
            && egg.y+eggSize > nest.y+nest.height/3
            && egg.y < nest.y+2*nest.height/3)
            {
                alert("Thanks for trying my profile widget. You will now be redirected to: \n"
                        + egg.url);
                location.href = egg.url;
                // remove egg from array
                eggs.splice(i, 1);
            }
    }
}

function updateUserInput() {
    if(!mouseEventFlag)
        return;
    
    for(var i in items)
    {
        var item = items[i];
        if( item.x < mouseEventX 
            && mouseEventX < item.x + iconSize
            && item.y < mouseEventY
            && mouseEventY< item.y  + iconSize)
            {
                /* instead of loading the egg in runtime, load it once (eggImage) at startup and use it.
                    Because for a web view, it takes a second to load the image, which looks bad.*/
                //var img = new Image();
                //img.src = item.egg;
                
                eggs.push({
                    x: item.x,
                    y: item.y + iconSize,
                    speed: eggSpeed,
                    image: eggImage,
                    url: item.url
                    });
                mouseEventFlag = false;
                return;
            }
    }
}

function drawBackground() {
    context.drawImage(background, 0, 0);
}

function drawWaves(){
    context.drawImage(waveImage1, wave1X, wave1Y);
    context.drawImage(waveImage2, wave2X, wave2Y);
    context.drawImage(waveImage3, wave3X, wave3Y);
}

function drawItems() {
    for(var i in items) {
        var item = items[i];
        context.drawImage(item.image, item.x, item.y);
        context.save();
        context.translate(item.x+10, item.y+10);
        context.rotate(-item.wingRotation);
        context.drawImage(wingImage, -wingSize, -wingSize);
        context.restore();
    }
}

function drawNest(){
    context.drawImage(nest.image, nest.x, nest.y);
}

function drawEggs() {
    for(var i in eggs) {
        var egg = eggs[i];
        context.drawImage(egg.image, egg.x, egg.y);
    }
}

canvas.onmousedown = function(e) {
    var mousePos = getMousePos(canvas, e);
    mouseEventX = mousePos.x;// - currentTranslationX;
    mouseEventY = mousePos.y;// - currentTranslationY;
    mouseEventFlag = true;
};

canvas.onmouseup = function(e) {
    mouseEventFlag = false;
};

function getMousePos(canvas, evt){
    // get canvas position
    var obj = canvas;
    var top = 0;
    var left = 0;
    while (obj && obj.tagName != 'BODY') {
        top += obj.offsetTop;
        left += obj.offsetLeft;
        obj = obj.offsetParent;
    }
 
    // return relative mouse position
    var mouseX = evt.clientX - left + window.pageXOffset;
    var mouseY = evt.clientY - top + window.pageYOffset;
    return {
        x: mouseX,
        y: mouseY
    };
}

Resources:
HTML5 Canvas Deep Dive
HTML5 Canvas Mouse Coordinates Tutorial

Qt Performance Tips

As I've been moving away from Qt, I thought I should write some notes about things I've been taking care of when writing a Qt application. Even if they are not the super-duper tips , they may be useful for some people out there.

These tips, and more you may know or apply by default, can make a huge impact on your application. As I remember, the right compination of these tips has helped me to reduce the time of a big operation, and I was shocked by numbers. Imagine a process that takes 7 milliseconds and a smooth UX  instead of 10~30 seconds and a laggy UX!

1- QCoreApplication::processEvents
When getting inside a long loop (for or while loop) the application will freeze because of the intensive resource consumption, so it is advised to process events inside the loops to allow other thread to run. You will notice the difference if, for example, you have a progress bar and you update it from inside the loop. The progress bar UI will not be updated eventhough its values have been set, because the loop is not giving the chance for other processes to work. A line like the following can do the job:

qApp->processEvents();

2- Timer slots instead of threads
Sometimes you want to make a non-blocking process inside your application, so you make another thread, then may get in the hastle of communication with the current thread or modifying the UI. An example I saw and liked was as follows:

QTimer::singleShot(0, this, SLOT(doSomething()));

It was used in a GUI application when there was much code to write in the constructor, and the code was making much delay in the appearence of the main window. So the solution was to move this code to a slot with zero timeout.
Another usage was when querying much data from a database, showing the results of the query simultiniously in the UI (e.g. filling a datasheet line by line) instead of waiting for the query to finish had a great impact on both performance and user experience.


3- QCache
One of the tasks I hate in my application is accessing a database or filesystem. So using the QCache class in my application could minimize the time of accessing a file or database. If you know how computer cache memory works on hardware level you will get the idea. When I query some data from database, I save them in QCache. So the next time I need the data, I look for it inside my cache member before going to the database. If I find it, I use it. I don't find it, I go to the database. This can help boosting the performance if you have an intensive usage of your database or filesystem in runtime (like a game data maybe or history suggestions). So what it the advantage of QCache over QMap or other classes? Here is a quote from the documentation:

When inserting an object into the cache, you can specify a cost, which should bear some approximate relationship to the amount of memory taken by the object. When the sum of all objects' costs (totalCost()) exceeds the cache's limit (maxCost()), QCache starts deleting objects in the cache to keep under the limit, starting with less recently accessed objects.

4- Qt Resource System
"The Qt resource system is a platform-independent mechanism for storing binary files in the application's executable". But it is not always the best choice when you have a lot of files. Remember that this increases the size of your executable, and memory usage. So when you have files that can be placed beside the app instead of beig inside it, this will make less memory usage, leaving more momory for later computations, especially when you have limited hardware resources. Keep the resource system for a minimal number of basic resources like, as the documantation says, a certain set of files (icons, translation files, etc.) that you don't want to run the risk of losing them.


5- Painting
When you paint some graphics inside the application, whether for a game or just a background, it is advisable to take care of the following.
a) You do not always need to repaint all the space. Sometimes, for example, you want to update a logo at the center of your window or splash screen, then why repainting hundreds of thousands of pixels when you only want to repaint a 64x64 rectangle in the middle?
b) When you repaint the same image over and over (for example, a background in the update() method), it is useless to load the image every time in a temporary variable inside the method because loading takes time. It is better to declare a local variable in your header file and load it once (in the constructor or anywhere when needed) then only repaint it without reloading the same image.


6- Database indexing
This may not be a Qt-specific tip, but it is still very important when you have a database of tables of 500,000 entries.

A database index is a data structure that improves the speed of data retrieval operations on a database table at the cost of slower writes and increased storage space. [Wikipedia]

If you do not know about this topic, I suggest you spent an hour reading and applying it on any database to notice the difference it makes.



7- You application in idle state
Some application features are only useful when the user is looking at them. For example when you are retrieving weather conditions from a remote server. What is the benefit of downloading an XML file, parsing, and updating your UI every couple of minutes when the user is not even looking? When the user is in a state when he does not benefit from a feature, it is better to stop/pause it.


So ... these were the performance tips I had in mind till now. I hope you learned something new from them.

Book Review: Google SketchUp for Game Design: Beginner's Guide

Today I'm writing a review on a very special book: Google SketchUp for Game Design: Beginner's Guide, by Robin de Jongh, Packt Publishing. I really enjoyed this book, and to be clear and organized, I'll write my review as definite points (as usual =) ).

Pros:

• As the book title mentioned, it is a guide for game design and beginners. So the book did not only go from bottom up in Google SketchUp, but also went to explain the basics of other applications that can get involved in the game design process like GIMP, and even introduced one of the best game engines: Unity.
• All tools and websites introduced are free, or have trial versions/accounts.
• The book author stressed on a very important thing: honesty, and the cover image is an example of this honesty. Because what you see on this cover is what you get and can do yourself using what you learned in Google SketchUp.
• The way the author explains is not boring or lengthy. But on the contrary, it was very smooth, interesting, and straight to the point.
• The author stressed on introducing the basic and important features of each applications without getting deep, which I considered it a very nice method in dealing with a beginner in any application. Beside some tips and tricks, and notes on how bigger companies do this work and manage such tasks.
• The book in general was like a quick tour in the world of assets and game design, opening some doors I might not see if I just go to learn Google SketchUp alone.

Cons:

• The websites and links provided inside the book were not all working. Some links were removed and some had different sub-directories, but somehow I managed to get to most of them by a quick search.
• The author sometimes missed some small details that -for a lazy beginner- can be crucial to get a step done. Some small details like the order of marking objects or how to use a certain tool. But the athor has indicated that this book needs some work, so there is no place for laziness.

All in all, I would recommend this book for reading, whether you are using Google SketchUp for game asset design, game level design, or even as an architect.