CPT Status Report 1 — May 29 2008
I am working for www.synctree.com for this Summer as an Intern. I have two accomplishments to share today :
# 1
I have had excellent learning experience this week. I have implemented a mini project in this week. It is available at http://office.synctree.com:3004/report/classifier_classification?cl=1 . It would take time to load as it is having huge number of images. I developed this entire interface, so that it becomes fast and easy to navigate through the website. I made is fast by replacing the web navigation by Ajax Requests. This is implemented in Ruby On Rails. I had no idea about Ruby on Rails when i was given this project. But i had an expert in the company who always showed me the path and helped me to understand it. He taught me the Directory Structure, Functions of modules involved and the basics of basic application. In summary, i learned Ruby on rails application development and also to add capabilities to existing code base.
# 2
Currently i am working on MapReduce. I was given a research paper http://labs.google.com/papers/mapreduce-osdi04.pdf to read up and understand the problems described in it. The paper illustrates Distributed Grep, Distributed Sort and many other highly time consuming algorithms. At Synctree there is a classification engine that has over a million images that have to be classified and many log files to be analyzed. This required many Hundreds of Machine hours to process.
What my boss intends to do is: to use MapReduce on a 8 node cluster available at Synctree premises to solve this problem. To get the hang of MapReduce i was given many similar problems to be solved on the board. My understanding of Mapreduce’s sucess is : It breaks down the processing of any program into highly modular and independent tasks. Mapreduce StandardLibrary gives the users of this library, an abstraction of Load Balancing, Inter task communication and many other issues that come into existence when the program becomes distributed and parallel
——————————————————————————————
CPT Status Report 2 — June 8 2008
I Added Video Support to Classifer Engine :
I worked on adding video support to the “classification engine”. I would give you a brief background about classification engine :: synctree.com builds social networking platforms for its clients, thus we have many social platforms to monitor and maintain. It is very crucial that we monitor users content, both images and video. Thus we build a classification engine that gives us a good UI to quickly browse through the contents and classifications done by amazon mechanical turk http://www.mturk.com/mturk/welcome .
Implement Sampling Algorithm and used ffmpeg sampler
Initially we had support only for the images as a part of my first project. But the second phase involved video support. I used ffmpeg http://ffmpeg.mplayerhq.hu/ for sampling flv files to images by sampling algorithm. Thus i had to call this program to sample images. The images that are obtained have to be associated with a video and to be shown as an overlay when you do a mouse over. The project is complete, but it will be migrated in the production environment by the end of testing phase.
We have started using http://rallydev.com/ project management tool. I am assigned a set of tasks every Monday for the entire week. These tasks are entered by the manager on Monday, after we discuss them in detail. Every day morning we have 20 min formal meeting where me and my colleagues report to each other on the development and what we accomplished the previous day. It is a good way as everyone has to say what they have done and achieved. Makes me feel pressured at times when it is 8 in the evening and i still have stuff to be completed. !! Great learning experience though.
——————————————————————————————-
CPT Status Report 3 - June 22 2008
Last 12 Days has been a roller coaster ride. Last Friday was the launch of fuel.tv . My main focus and assignment involved fixing bugs reported in the Fogbugz.com for fuel.tv . Majority of bugs involved changes to be made in CSS and JavaScript. Each browser works in different way, thus the code had to be made in such a way that even the minor alignment and incompatibilities is taken care of before the deadline. The work was intense as the deadline was coming fast and bugs reported by the Q&A team kept increasing. I also got introduced to Blueprint CSS framework while i was doing this debugging. It is an excellent way in which one can use CSS.
I also got a feedback from My Boss after one month of Job. I was given very constructive feedback about my performance. I was told very frankly that, if i wanted to continue with this job in Fall, i would have to Ramp up my productivity and focus. My boss told me the distinction between a Cook and a Chef, he told me, he was more interested in making me a Chef rather then a cook in Software Engineering. I must say, I am fortunate to learn things under his guidance.
The next important achievement is on successfully installing ORACLE 11 G on Unbuntu Linux operating System. It took me 2 full days to figure it out and make it working. I went through lot of forums online and read lots of How to Tutorials. One thing i realized is that : working on linux made me think in much detail, and it also made my understanding of the software’s requirement. In XP it was just with a click of a button, and the software is designed in such a way that installation dependencies does not come into play. In Linux its like resolving all those dependencies and building the tree. It is more intuitive now as I installed the software.
——————————————————————————————-
CPT Status Report 4 — July 10 2008
1. Installation of Server with Fedora Core 7 : I was given an extra server that we had at the office to install Oracle 10G. I had installed ORACLE 11G on my laptop, but realized that it required at least 2GB Ram. I formatted the Server and hooked it up to the network. Used Fedora Core 7 boot Disk and used HHTP for installation. I also learned to find out mirrors that are near to you. Gave the Server local address. Did the NAT mapping on port 10000-10999 for the server. Installation of Oracle 10G on Fedora Core 7 : Did a remote Oracle 10G installation using SSH. Many config files were corrupted as Oracle does not support Open Source Software. A little bit of tweaking, made the database server run.
2. Got introduced to YAhoo Pipes. I wrote a Perl script to automate the Blog Generation on the blnk.com platform. The script checked for duplicate blogs and only inserts new blogs from the feed generated from Yahoo Pipes.
3. Field Trip to Cogen Data Center - Got introduced to the Servers the company owned, found their IP addresses and labled them on the server.
———————————————————————————————————–
CPT status Report # 5
Last 10 days was full of learning many things. Following is the list of things that I learned
and applied::
1. FOX.com Project: Am working on a JSP project for FOX. FOX has got its own
customized CMS system. They want us to make a JSP website such it works off the
URLS that they expose. The following are the deliverables of the project:
Interfacing with the Fox Sports CMS:
* We have broken each page up into 4 sections (these will be the variable parts of the
content; or what can be changed on the Fox side through the CMS).
* For each section we will be using a simple remote resources class which will fetch
the content based on the section.
* Each section will be mapped to a specific URL using a java properties file, which will
be included in the distribution (it will be inside the war file).
* We will be using Ant build tool.
2. Prototype and Scriptaculous (Javascript Frameworks):
I learned the basics as of now. I also got an overview of all the available functions
that were built in. It does provide an excellent way of programming in browser
independent manner. There are also many Ajax features that the frameworks provide.
3. Blueprint CSS Framework
This is another project which was used in the platform that our company uses. It
provides for a strong CSS formatting. I used to sample page. That gave me hands on
experience.
I also launched tech.kunalcholera.com which focus on my Technical abilities.
I have 3000 + pageviews on my blog blog.kunalcholera.com last month. With 15 unique
visitors every day (Google Analytics), as of now..
——————————————————————————————–
CPT Status Report 6
This is my CPT status report # 6.
This is my last status report.
This week was very productive. My main task involved using PERL APIs
which were developed for the platform. These APIs expose all services
of the social networking platform. Using this service, one can do
everything on the platform.
I developed a perl project, that imported the database of
www.hbeshafriends.com to the local platform compliant database. It
involved developing customized scripts for various database access and
an API calls.
The main components for all the scripts were :
1. Database Access in perl.
2. API structure understanding.
3. POST request generation.
4. Hash Map generation in Perl.
5. Response generation obtained from API call.
6. Keeping count of success and failed API transaction call.
——————————————————————————————–
CPT Status Report # 7
Fox Sports Mobile Launch : http://s220.rack0.servers.synctree.com/FoxSportsMobile/
I was part of the making of above web site for FOX.com. It involves JSP and we are using Ant build tool for deploying to
tomcat server. There were many phases of implementation of this project.
1. Creation of Mockups - Set of Designers produce an image of how the web site would look.
There were three rounds before a final conclusion was reached for the design.
2. Creation of HTML Templates : This job is out sourced to a company in India.
3. Creation of JSP functionality : This was my job. I had to integrate the web site with the CMS provided by Fox.They expose all the data on the website to their CMS. This CMS would generate URLs which can be exposed. I created Java class with functions to obtain URL body as String and added display logic to it.
4. External Flash movies and Flash Object navigation was provided by FOX for direct Embed.
Fox Mobile Emulator :
1. Developed this proxy http://s220.rack0.servers.synctree.com/FoxSportsMobile/MobilePhone.jsp?link=http://sports.mobile.msn.com/en-us/ that goes into this page : http://s220.rack0.servers.synctree.com/FoxSportsMobile/fsm_web.jsp .The actual page used is http://sports.mobile.msn.com/en-us/ .
What this proxy does is the following : 1. Extracts all links from http://sports.mobile.msn.com/en-us/ and makes them link to my proxy.
2. It works for all website. like for example : http://s220.rack0.servers.synctree.com/FoxSportsMobile/MobilePhone.jsp?link=http://www.google.com
3. It gives users of using a site which may embed and give other users the same experience of navigation the original site.
4. This proxy setup will be used in future by blnk.com to monitor user searches and provide better results by modifying the search in the background. This would also give us a good understanding of what the users of the social networking platform are looking for. I have used Prototype javascript framework for doing this in cross browser compatible manner.
It is fun working on this project, but it becomes very irritating when the client constantly keeps changing his mind on what he wants. It make the programmers slow in implementation. Its like this : We had a talk with FOX manager for mobile division for 2 hours about any more modifications that he required. I had around 45 more requirements after the launch. My boss motivated me to get it over with. I sat till 10pm and completed it. The next day we get an email from FOX saying about his new mindset. This makes me rather wait for the client to settle down on their requirements.
——————————————————————————————–
CPT Status Report # 8 — September 6th 08
I have focused last ten days in coordinating with FOX.com to give it the finishing touch. The clients keep changing their mind over what they really want.
I have worked on Intercasting Corp Project the remaining part of the week :
Intercasting Corp : Exposing APIs of the current social networking platform to Intercasting Corp to make BLNK.com available on mobile phones.
Intercasting Corp provides mobile platform for social networking platfroma. This could bring huge revenue to the company. 75 Cents per month per user. 30,000 out of 1,000,000 users expected to subscribe. My job is to expose all the social networking platform functionality that we have as API calls. Thus when all APIs are exposed, Intercasting Corp would make a call to those APIs that out platform would be able to support. We will generate an API key for Intercasting Corp so that only they can make those URL calls. Task is based on ruby on rails platform. Estimated time of completion is 1 week.
Next week I will be working on Amazon EC2 and Amazon S3. We plan to move the platform to Amazon data centers. This would help us bring down the cost and also help us to scale for the new projects that are upcoming in the company.
During the completion of FOX project, I learn t many things from my mentor, who taught me the
Importance of writing Clean Code ::
My mentor whenever explained me examples of codes. He always made it a point to meticulously write the code. He took extreme care when he aligned the code. He also added snippets of explanation. He told that a well documented code, helps in many ways.
I took out the following points during our interaction :
1. For the clients to understand and solve some simple customizations that they want over period of time.
2. Easy to debug, when ever he got some bugs, he could figure out the solution very quickly.
3. It also becomes easy to add functionality when the code is made modular.
4. He explained the importance of modularity in every possible situation. Modular code is compatible with all code layers added on top of it. It also becomes one point of change. Thus modular code helps in making changes.
5. Estimation of Client problems : A clean code would make functions such that, it becomes obvious to the client to look for at certain place. Use of camel case class functions make it easier for the software programmer to understand it.
6. Consistent Naming Conventions : He advised me to have consistent naming conversions through out the project. Having consistency in the naming conventions makes it easier for browsing the code when that code has to be debugged.
7. Maintaining meaningful folders and placing codes in a proper directory is also helps when you have to fix something you don’t know about.
—————————————————————————————–