Reflecting on GSoC
The summer has ended for most of us and I'm glad to say that lots of progress has been made since my last entry. Devcathlon has given me lots of ups and downs in the past week. After successfully building the system with a persistent layer using Hibernate, many items such as the UI and game functionality still needed to be dealt with. Although not all objectives were accomplished in the last iteration of Devcathlon, I did have a chance to touch face with many of the issues that will need to be resolved in the next upcoming release of Devcathlon. They seem minor but can sometimes prove to be quite tedious and lots of code digging. This summer's venture has been the most time consuming, yet an overall rewarding experience. I felt that this program has given me greater insight into what development is in a larger based environment. The project at times could get very lonesome, though I did find ways to get motivated again. Many long afternoons and late nights have given me a real appreciation for time. During the last week of GSoC, I granted myself "time" to dedicate myself to the last moments of Devcathlon for this summer. It was difficult since my coding was usually compressed into one weekend which consisted of two days and a lot of distractions. The difficulty of clearing my mind from one project to the other proved to push me to my limitations. Something had to give; so I gave my last focus this summer to Devcathlon (I know it should have been my first and foremost focus). I know that it's too late to turn back the clock, but I've learned a valuable lesson and something about myself along the way.
Devcathlon's beginnings
The starting of Devcathlon was a very hodgepodge of wicked Wicket fun. People were still getting used to the framework's concept of MVC and Wicket's flexibility and support of components. The building blocks of Devcathlon centered around exploring these new grounds. I felt that this Summer was to pay homage to what I've learned from using Wicket and my previous encounters with other frameworks. I did incorporate changes to the workings of the profile and team sections, but others were left out due to time constraints. Using models is a must in the MVC framework. And using them correctly shows that you really know your stuff. I had to spending many moments re-reading the chapter on models and detachable models. They are very important in the development process and I wished it had been used more frequently in Devcathlon's 1.0 release. Also reusing views and controller functions ensure that you are also being DRY. I could see that some of those concepts were being injected late into Devcathlon, but others were still very traditional. I honestly saved what I could, and some were forgiven.
Working with Devcathlon
When getting along with an existing system (like Devcathlon) which had multiple people before you developing; you tend to see bits and pieces of the puzzle that fit together. Once you fit those pieces together then the next piece becomes a search. I felt that many of the things I didn't understand from the beginning was an inevitable search to find that missing link. After a while I got used to seeing the patterns people made in their code and what they would often do. I would also tend to have my opinions on certain paths people took when coding. I can't say that Devcathlon was entirely me, because it isn't. For this reason, it was difficult to make hasty changes to the code base. I wanted to respect the code done before me even though some things could have been done better and much more efficiently. I'll suggest some later, but the point is that Devcathlon had a personality to it as do any open source project out there. I've begun to notice how difficult it is for teams to collaborate and come to terms with what is standard.
Technical Review of Devcathlon (pre v2.0)
If you've got a class (model) then use it!
There are many instances when creating a form that I would see no models ever being used. Variables that can be easily referenced using the binding of an existing class model would have relieved the purpose of recreating them again in the form class.
Panels are great, but don't go overboard!
Following the DRY concept, one could really go overboard into drilling everything down to a label component is reusable and can be turned into a panel. Unfortunately this could lead to unnecessary growth of files which would make it much harder to manage. Just know that things can always be re-factored eventually after knowing that they are needed in more than one place.
Keep our styles clean yet explicit.
CSS files were all over the place and some still are and will need to be removed. A little helpful note on what the style pertains to would give nice clues as to where to find it in the markup. I found this to be difficult when certain styles were overriding others. Please make up you mind!
Find the project, here along with all the main updates: http://code.google.com/p/hackystat-ui-devcathlon/
Closing thoughts
The overall experience was quite a journey from start to finish. I had an absolute pleasant time, keeping my mind busy this Summer. Hopefully as school starts my mind will start to cool off... NOT! ;)
Also many thanks to my mentors who tried desparately to help motivate me and to keep me on track. Apologies to anyone I may have offended during the course of this Summer (I meant every word). And I wish the best of luck to future GSoCers.
Long live the code and tradition! Cheers!
Monday, August 24, 2009
Wednesday, August 12, 2009
[GSoC09] Overcoming a hurdle
If you were wondering why I haven't been blogging lately, then it's because I hit a "wall." I thought after taking the first few blocks off I would be free of trouble, another layer gets built, but only stronger and thicker than the last. I have to say that this project, in addition to this summer's workload has had a profound impact on my performance as a programmer. I've essentially realized my limits and as usual would draw insight from these experiences.
So I cleared my head this past week and started to extract what I had done on the branch copy (for db development) and slowly transitioned to the main interface in the trunk of our repository.
Below are some notes for myself and to future developers of Devcathlon:
Issues building a DB with a pre-existing system designed by multiple users:
These are the issues I ran into with Devcathlon that seemed to really give me the most problems.
The integrity of our DB:
There were many places in the application where (when we would test) that checks were made by the DB and not by the application before hand. This was mainly due to how DevcathlonTestHelper was built, to create and immediate delete a record, without concerning itself with other dependent constraints. In some instances, this would occur while testing team resignations which required two teams with a user being the team owner of one team and a member of the other. So by deleting one of the teams, this would cause a constraint error which is totally valid but not handled by our initial test helpers. Modifications were needed in this area.
*A special note on a test db: It would be beneficial to setup a Devcathlon test db for creating and dropping tables while developing in Devcathlon. This would keep your tests cleaner and ensures the integrity of all records.
In addition to changes to the DevcathlonTestHelper, the DevcathlonConfiguration needed to be initialized for all events defined in the configuration.xml file. This was needed in order for all event subclasses to properly map with the abstract event class upon declaring any particular event. Errors would occur when an event was not initialized by the ConfigurationManager.
The other problem relied on the fact that certain classes needed to be mapped (mirrored) that were originally built via JAXB. I.e.: The Project JAXB generated class needed to be mirrored in Devcathlon and renamed as TeamProject for persistence.
Thankfully seeing the "BUILD SUCCESSFUL" message (noted @ 5:45am this morning) appear after running 'ant -f verify...,' I finally felt a large weight lifted off of me. I'm now happy to know that the DB is settled and I'll be able to sleep better tonight.
This week:
Test events and single matches (08/19-20)
Work on UI improvements (08/19-21)
Clean-up old wicket panels (08/19-21)
Documentation: Installation & Issue notes (08/22)
So I cleared my head this past week and started to extract what I had done on the branch copy (for db development) and slowly transitioned to the main interface in the trunk of our repository.
Below are some notes for myself and to future developers of Devcathlon:
Issues building a DB with a pre-existing system designed by multiple users:
These are the issues I ran into with Devcathlon that seemed to really give me the most problems.
The integrity of our DB:
There were many places in the application where (when we would test) that checks were made by the DB and not by the application before hand. This was mainly due to how DevcathlonTestHelper was built, to create and immediate delete a record, without concerning itself with other dependent constraints. In some instances, this would occur while testing team resignations which required two teams with a user being the team owner of one team and a member of the other. So by deleting one of the teams, this would cause a constraint error which is totally valid but not handled by our initial test helpers. Modifications were needed in this area.
*A special note on a test db: It would be beneficial to setup a Devcathlon test db for creating and dropping tables while developing in Devcathlon. This would keep your tests cleaner and ensures the integrity of all records.
In addition to changes to the DevcathlonTestHelper, the DevcathlonConfiguration needed to be initialized for all events defined in the configuration.xml file. This was needed in order for all event subclasses to properly map with the abstract event class upon declaring any particular event. Errors would occur when an event was not initialized by the ConfigurationManager.
The other problem relied on the fact that certain classes needed to be mapped (mirrored) that were originally built via JAXB. I.e.: The Project JAXB generated class needed to be mirrored in Devcathlon and renamed as TeamProject for persistence.
Thankfully seeing the "BUILD SUCCESSFUL" message (noted @ 5:45am this morning) appear after running 'ant -f verify...,' I finally felt a large weight lifted off of me. I'm now happy to know that the DB is settled and I'll be able to sleep better tonight.
This week:
Test events and single matches (08/19-20)
Work on UI improvements (08/19-21)
Clean-up old wicket panels (08/19-21)
Documentation: Installation & Issue notes (08/22)
Sunday, July 12, 2009
[GSoC09] Devcathlon UI and other random thoughts
Past week:
This week was more targeted in testing and running Devcathlon's game events. I got a chance to look over a few of the events as well as make improvements to the user interface. I honestly had more fun with the UI, since it gave me the chance to be creative. Also this past week was mostly angled towards packaging for GSoC's mid-term release.
UI Improvements:
The first time you visit Devcathlon, you are presented a functional, yet not very pleasing UI that included a conglomerate of panels and links that semi-worked correctly. I decided that if I needed to start somewhere, then it had to be this very initial page. From here, I devised a new scoreboard, that updates frequently as the visitor sits back and watches things change on screen. It's more dynamic, much more intuitive and incredibly fun to look at.
In the process of creating this component, there are some things that were notably noted for future reference and eventually documented.
Dependencies:
Wicket's date-time package extension requires a tool called, Joda. It took a while to figure out the reason why the page could not initialize when coding a date-time picker with the following:
Igor was quick to notify someone else with the similar problem by concluding that it would be easier to use a project management tool, called Maven. It's probably the most reasonable thing to do in the future since Wicket projects are now more geared towards using this to properly manage project builds, documentation and resolving dependencies. However, Devcathlon is quite comfortable now in just using Ant as a building tool. Ant serves to be flexible and less standardized than Maven, yet Maven does enforce good practices in folder structures and offers other great project organizations.
** (07-2009) UPDATE - All projects for Hackystat will be switched to integrating with IVY soon.**
This week:
- Resolve issues with mapping JAXB classes via HyperJAXB. (Complete DB)
- Setup and install Devcathlon to public (test) server
- Test game events and pull live data
- Incorporate new UI
Other notes:
Since other tools were needed to properly build the project through our continuous integration, via Hudson, I didn't want to risk failing a build. I'm keeping things in-house with my mentors until Monday when we set up a proper test server where I could install exactly what I need and want on it. Currently testing will be done by one user (myself) and eventually by my fellow Hackystat GSoCers.
I also mentioned earlier last week about switching to Netbeans 6.7, but my project mentor, Philip suggested that I should stick with Eclipse since we currently only have one sensor that supports Eclipse. On that note, this would make for a great plugin idea for Netbeans to support sending Hackystat sensor data via the IDE. Hopefully someone with the time could venture this avenue.
This week was more targeted in testing and running Devcathlon's game events. I got a chance to look over a few of the events as well as make improvements to the user interface. I honestly had more fun with the UI, since it gave me the chance to be creative. Also this past week was mostly angled towards packaging for GSoC's mid-term release.
UI Improvements:
The first time you visit Devcathlon, you are presented a functional, yet not very pleasing UI that included a conglomerate of panels and links that semi-worked correctly. I decided that if I needed to start somewhere, then it had to be this very initial page. From here, I devised a new scoreboard, that updates frequently as the visitor sits back and watches things change on screen. It's more dynamic, much more intuitive and incredibly fun to look at.
In the process of creating this component, there are some things that were notably noted for future reference and eventually documented.
Dependencies:
Wicket's date-time package extension requires a tool called, Joda. It took a while to figure out the reason why the page could not initialize when coding a date-time picker with the following:
private final Date date = new Date();
...
DateTextField dateTextField = new DateTextField("dateTextField", new PropertyModel(this, "date"), new StyleDateConverter("-S", true));
...
form.add(dateTextField);
dateTextField.add(new DatePicker());
...
Igor was quick to notify someone else with the similar problem by concluding that it would be easier to use a project management tool, called Maven. It's probably the most reasonable thing to do in the future since Wicket projects are now more geared towards using this to properly manage project builds, documentation and resolving dependencies. However, Devcathlon is quite comfortable now in just using Ant as a building tool. Ant serves to be flexible and less standardized than Maven, yet Maven does enforce good practices in folder structures and offers other great project organizations.
** (07-2009) UPDATE - All projects for Hackystat will be switched to integrating with IVY soon.**
This week:
- Resolve issues with mapping JAXB classes via HyperJAXB. (Complete DB)
- Setup and install Devcathlon to public (test) server
- Test game events and pull live data
- Incorporate new UI
Other notes:
Since other tools were needed to properly build the project through our continuous integration, via Hudson, I didn't want to risk failing a build. I'm keeping things in-house with my mentors until Monday when we set up a proper test server where I could install exactly what I need and want on it. Currently testing will be done by one user (myself) and eventually by my fellow Hackystat GSoCers.
I also mentioned earlier last week about switching to Netbeans 6.7, but my project mentor, Philip suggested that I should stick with Eclipse since we currently only have one sensor that supports Eclipse. On that note, this would make for a great plugin idea for Netbeans to support sending Hackystat sensor data via the IDE. Hopefully someone with the time could venture this avenue.
Friday, July 3, 2009
[GSoC09] Closing into Devcathlon v2.0
Premise:
In this blog entry, I'll point out the improvements and near completion of Devcathlon v2.0. I know that mid-term evaluations are coming up for GSoC, so I'll need this week to concentrate on the core features. The things I hope to accomplish from now on are almost purely aesthetic, along with minor tweaks to the functionality here and there. I must apologize for my absence last week due to some personal health issues.
Hibernate and me:
For the last two weeks, I've been working on the database through the use of Hibernate. I got a chance to encounter some pretty neat, hidden surprises along the way; one of which deals with performance issues, due to the fact that Hibernate is so "lazy." Hibernate uses this to preserve memory space by not loading the entire objects from the database, but rather proxies at run-time instead. So I ran into some Lazy initialization errors that seemed very annoying at the time. For simplicity, I decided to lift the burden by just disabling this feature on certain mapped entities. However, I did take the precaution to apply some fetching strategies (join rather than a select) for an implicit get() or load() data retrieval. Overall the database is coming along, I did have some questions about class entities that require mappings, such as generated classes from JAXB, Project.class. This is required by most of the other class models (Team and Match), and I'm wondering if the class would need to be moved out as a User defined Type (by extension of Hibernate's UserType) or a custom subclass. Since this is a generated Java class from XML, I'm afraid to take either steps. Considering the time I have left, and mid-term evaluations are right around the corner, I've decided to move onto the UI goodness.
The Good Stuff:
I went back to some elements of our first designs of Devcathlon and took the time to rethink about certain areas. Everything is starting to come into perspective, as far as the design goes and we will be seeing some evidence of this in the following 2.0 release. Below is a list of in-progress features being worked on:
Key features:
Search engine (browse for entities: User/Profile, Team, Match)
Calendar selection
Improved Scoreboard main page - total revamp with live updated listings
CSS improvements to inherited Panels and Base pages.
Trying something new:
At the mist of an update, my Eclipse IDE went a-wal. It just stopped liking itself, or rather hating me... I had difficulty with start-ups, error messages would pop-up saying NoClassDef, and editors could not open up anymore. I got frustrated with trying to remedy this problem, since it has happened before. I like Eclipse and all, but It has given me a history of some disappointment. I can't blame it entirely, since I know that most of this is due to the fact that I install other third-party vendors (plugins) in Eclipse. This is the reason why I made my recent move over to using NetBeans 6.7. In all honesty, Eclipse is perfect for a lot of situations but it calls for other resources that might not be perfectly integrated with itself natively. On the other hand, NetBeans almost feels a lot like developing on an enterprise IDE, like MS Visual Studios, etc. It has all the languages I develop in on a daily basis built-in, offers additional services like connecting to databases, creating custom web services, and most importantly it's still FREE.
As for racking up dev-time, I'll have to resort to using Hackystat's Tickertape UI for now.
Conclusion:
In the final mid-term week of GSoC, I'll attempt to update the documentation, make improvements to the UI and ready the package off to my mentors. It's been quite a ride and I hope this project satisfies a need to many developers and project managers.
In this blog entry, I'll point out the improvements and near completion of Devcathlon v2.0. I know that mid-term evaluations are coming up for GSoC, so I'll need this week to concentrate on the core features. The things I hope to accomplish from now on are almost purely aesthetic, along with minor tweaks to the functionality here and there. I must apologize for my absence last week due to some personal health issues.
Hibernate and me:
For the last two weeks, I've been working on the database through the use of Hibernate. I got a chance to encounter some pretty neat, hidden surprises along the way; one of which deals with performance issues, due to the fact that Hibernate is so "lazy." Hibernate uses this to preserve memory space by not loading the entire objects from the database, but rather proxies at run-time instead. So I ran into some Lazy initialization errors that seemed very annoying at the time. For simplicity, I decided to lift the burden by just disabling this feature on certain mapped entities. However, I did take the precaution to apply some fetching strategies (join rather than a select) for an implicit get() or load() data retrieval. Overall the database is coming along, I did have some questions about class entities that require mappings, such as generated classes from JAXB, Project.class. This is required by most of the other class models (Team and Match), and I'm wondering if the class would need to be moved out as a User defined Type (by extension of Hibernate's UserType) or a custom subclass. Since this is a generated Java class from XML, I'm afraid to take either steps. Considering the time I have left, and mid-term evaluations are right around the corner, I've decided to move onto the UI goodness.
The Good Stuff:
I went back to some elements of our first designs of Devcathlon and took the time to rethink about certain areas. Everything is starting to come into perspective, as far as the design goes and we will be seeing some evidence of this in the following 2.0 release. Below is a list of in-progress features being worked on:
Key features:
Search engine (browse for entities: User/Profile, Team, Match)
Calendar selection
Improved Scoreboard main page - total revamp with live updated listings
CSS improvements to inherited Panels and Base pages.
Trying something new:
At the mist of an update, my Eclipse IDE went a-wal. It just stopped liking itself, or rather hating me... I had difficulty with start-ups, error messages would pop-up saying NoClassDef, and editors could not open up anymore. I got frustrated with trying to remedy this problem, since it has happened before. I like Eclipse and all, but It has given me a history of some disappointment. I can't blame it entirely, since I know that most of this is due to the fact that I install other third-party vendors (plugins) in Eclipse. This is the reason why I made my recent move over to using NetBeans 6.7. In all honesty, Eclipse is perfect for a lot of situations but it calls for other resources that might not be perfectly integrated with itself natively. On the other hand, NetBeans almost feels a lot like developing on an enterprise IDE, like MS Visual Studios, etc. It has all the languages I develop in on a daily basis built-in, offers additional services like connecting to databases, creating custom web services, and most importantly it's still FREE.
As for racking up dev-time, I'll have to resort to using Hackystat's Tickertape UI for now.
Conclusion:
In the final mid-term week of GSoC, I'll attempt to update the documentation, make improvements to the UI and ready the package off to my mentors. It's been quite a ride and I hope this project satisfies a need to many developers and project managers.
Monday, June 22, 2009
[GSoC09] Hibernate + Devcathlon
Premise:
This past weekend I had the chance to dive into implementing Devcathlon's database back-end (Derby) by using Hibernate. I, unfortunately ran into some issues while applying these changes. They seem very pin-pointed since changes were recently made using Hibernate, but there were other confounding factors that hindered a smooth application. After last week's implementation, I was sure that the issue seemed small and easily doable. However the common, 1001 timeout error (which occurs during a Devcathlon's call out to Hackystat's SensorBaseClient) proved to be more of a challenge than I thought. In this entry I'll describe the issues I ran into, and the many solutions leading up to resolving this problem. I'll also mention about next week's goals and a timeline for Devcathlon's first milestone deliverable.
What was the issue?
SensorBaseClient 1001 timeout error - This happens due to the increase complexity of the application and heavy call loads out to Hackystat's SensorBaseClient server. This occurred while running the JUnit test cases against recent modifications while applying Hibernate. Another possible problem may arise due to the increase size of the application, which causes an overload memory usage on the contained JVM's heap size.
Resolving the issue:
I went through dismantling many of the previous code implementations. It wasn't awful, though I felt that changing the current state of the application was too obtrusive. I was prone to just keeping my changes reserved to just my additions. Any modifications to the actual code base would need to be reasonably thought out. So the (temporary) remedy for this matter happened to reside in modifying the DevcathlonTestHelper class. The error mostly occurs while trying to make multiple calls out to the SensorBase, along with calling out to Hibernate. The calls to Hibernate happens to stall (exactly at the same spots) and the connection to the SensorBaseClient apparently times out. I tried various ways to remedy the problem, a few of which that were clearly suggested by Johnson in this videocasting. If you ever run into this error, I would suggest to first watch that video and try out each solution, if all else fails then it's safe to assume that the problem must be you! The work around was applied yesterday, and I was thrilled that the build finally passed verify. The code has been unloaded to SVN and I would like some kind of code review from my mentors.
Changes for this past week.
r903 & r904
Next week's work load:
Just a reiteration, the primary goal at this point is to clearly implement a DB for Devcathlon. Considering how much I've endured this past week, It'll hopefully support a faster development time, now identifying those issues. Along with finishing the database, I will get the ball rolling next week for the user interface.
The first delivery date will be within two weeks from now, which will include a solid database implementation using Hibernate (with all/any kinks worked out), UI improvements mentioned in earlier blog posts and suggestions made by peers, and finally a mailing system using JavaMail. Look out for the official beta release of Devcathlon v2.0 on July 6, 2009.
This past weekend I had the chance to dive into implementing Devcathlon's database back-end (Derby) by using Hibernate. I, unfortunately ran into some issues while applying these changes. They seem very pin-pointed since changes were recently made using Hibernate, but there were other confounding factors that hindered a smooth application. After last week's implementation, I was sure that the issue seemed small and easily doable. However the common, 1001 timeout error (which occurs during a Devcathlon's call out to Hackystat's SensorBaseClient) proved to be more of a challenge than I thought. In this entry I'll describe the issues I ran into, and the many solutions leading up to resolving this problem. I'll also mention about next week's goals and a timeline for Devcathlon's first milestone deliverable.
What was the issue?
SensorBaseClient 1001 timeout error - This happens due to the increase complexity of the application and heavy call loads out to Hackystat's SensorBaseClient server. This occurred while running the JUnit test cases against recent modifications while applying Hibernate. Another possible problem may arise due to the increase size of the application, which causes an overload memory usage on the contained JVM's heap size.
Resolving the issue:
I went through dismantling many of the previous code implementations. It wasn't awful, though I felt that changing the current state of the application was too obtrusive. I was prone to just keeping my changes reserved to just my additions. Any modifications to the actual code base would need to be reasonably thought out. So the (temporary) remedy for this matter happened to reside in modifying the DevcathlonTestHelper class. The error mostly occurs while trying to make multiple calls out to the SensorBase, along with calling out to Hibernate. The calls to Hibernate happens to stall (exactly at the same spots) and the connection to the SensorBaseClient apparently times out. I tried various ways to remedy the problem, a few of which that were clearly suggested by Johnson in this videocasting. If you ever run into this error, I would suggest to first watch that video and try out each solution, if all else fails then it's safe to assume that the problem must be you! The work around was applied yesterday, and I was thrilled that the build finally passed verify. The code has been unloaded to SVN and I would like some kind of code review from my mentors.
Changes for this past week.
r903 & r904
Next week's work load:
Just a reiteration, the primary goal at this point is to clearly implement a DB for Devcathlon. Considering how much I've endured this past week, It'll hopefully support a faster development time, now identifying those issues. Along with finishing the database, I will get the ball rolling next week for the user interface.
The first delivery date will be within two weeks from now, which will include a solid database implementation using Hibernate (with all/any kinks worked out), UI improvements mentioned in earlier blog posts and suggestions made by peers, and finally a mailing system using JavaMail. Look out for the official beta release of Devcathlon v2.0 on July 6, 2009.
Friday, June 12, 2009
[GSoC] Derby race to the finish...
Purpose
In my last posting, I mentioned about ditching BDB for Apache Derby DB as Devcathlon's database back-end. I'm glad to say that that was a reasonable choice and most likely a better one in the long run. For this entry, I'll mention about my progress with the application and what I have in store for the next few weeks.
Starting out...
At first I decided to create the database using traditional SQL dialects for building the tables necessary for Devcathon. Below is the necessary Derby configurations and the resulting db schema.
Derby configuration:
Installed and included an environment variable for Derby.
Created DevcathlonDerbyClient.java class (within the same directory of Devcathlon's Jetty-server Start class). The class includes the connection details and initialization of Devcathlon's database. It was implemented using Derby's included JDBC client driver, ClientDriver class. Next I included the client driver for the start of Devcathlon's application in Start.java. Then I started Derby's Network Server, (ran startNetworkServer[sh|bat]) on port 1527. The application fired fine, and Devcathlon was able to communicate successfully with its database.
SQL DB Schema 'Devcathlon':
The full schema file can be found, here. This includes my quick implementation of Devcathlon's database schema for the following tables: Profile, Team, Project, Match and Event.
For convenience, the file is read in by the DevcathlonDerbyClient driver, then parsed and finally executed to initialize the database.
After spending some time reading up on ORMs, I decided to give it a try. I last mentioned about ORMs in my last few postings; this week was the real test and eventual implementation of an ORM, called Hibernate. I had a few other choices for a JPA + EJB standards (OpenJPA), but I think that Hibernate makes for an advanced extension to these APIs. Considering the immense support (online tutorial/manual) and popularity (in forums), I was able to get up and running with Hibernate within less than an hour.
Hibernate-Devcathlon configuration:
I downloaded the distribution (v3.3.1) and extracted it into to my local namespace. I also included an environment variable that links to the root directory of Hibernate, called HIBERNATE_HOME. Next, I modified the Ant build.xml to include Hibernate's required jars for compilation. Then I created a Hibernate configuration file, called hibernate.cfg.xml. The configuration file defines the SessionFactory settings, which includes the database connection(s), hibernate.cfg.xml files, pool connectivity, and debug options. Below is one possible rendering of an hibernate.cfg.xml file for Devcathlon. *Note that Devcathlon will default to looking for this configuration file under the ~/.hackystat/devcathlon/db/hibernate/hibernate.cfg.xml. This is intended so that system administrators can define their own configurations that may require a different database, or whatever is most optimal to their settings.
Loading this configuration file, required a basic HibernateUtil.java class to handle creating thread-safe, session-per-request SessionFactory. This class is intended to handle initializing one global session, providing pool connections for less overhead on the database, and closing a session. Most importantly, this will be the main communication between our data objects and Devcathlon's database. *Note on property, "hbm2ddl.auto" will be switched to a value of "create-drop" for testing purposes only. On an actual production server, this configuration file should just include a "create" value (without the quotes) for permanence.
Since I started out with building the db schema manually, it really benefited in creating the relational mappings for Hibernate. I could have gone with Hibernate's bottom-up approach by reverse engineering the process to get generated (Java annotated) mappings, but I wanted to see how close my schema was compared to the Hibernate's interpretation of Devcathlon's related classes, and mainly since all the Java persistent classes (ordinary POJOs) have already been built. Thus this is more of a top-down strategy that requires developers to build these relational mappings first, and finally have Hibernate auto-generate and apply its interpreted database schema.
Structural pattern to defining and applying relations:
Define the relational mapping of a persistent class entity, ex: User, Profile, Team, etc.
Add these relational mappings (*.hbm.xml) to the session configuration, by simply adding them programmatically in HibernateUtil.java.
All *.hbm.xml relations-per-entity will reside next to their declared entities and within the same folder.
Creating first relational mappings:
For initial testing purposes, I applied my first relational mappings for the User-Profile class models. The two class models have a one-to-one relation, or simply explained: A User has-one Profile and a Profile belongs-to a User. For a technical explanation, a Profile model would contain the primary key constraint, which 'references' the User model. The relationship will be represented as a bi-directional relationship, for equal referencing by both models.
Below is one way of creating the relational mappings through Hibernate. I could have also followed JPA standards and included some supportive Annotations, but that would eventually become too obstrusive (adding annotations) and more work (removing annotations) if we ever decide to use another ORM. Plus, using these xml configuration files is simple and easy to interpret.
Defining the user.hbm.xml:
Defining the profile.hbm.xml:
Applying the db store:
Since Devcathlon's folder architecture is quite organized, and well thought out for future implementations of a database in mind. It was almost an effortless job on my part, considering that all I needed to do was send a create, read/load, update or delete to Derby's database. Each CRUD function was beautifully built and implemented within each entity's engine manager. So rather than storing everything in the Devcathlon session store, everything gets applied to Hibernate's SessionFactory. Below is a usage template for applying this SessionFactory:
I ran into some issues recently after running JUnit tests with my implementation, caused a SensorBaseClient timeout 1001 error. The error only occurs while running 'testNoData' for the very first test Event implementation (TestBestCoverage). It's not an intermittent issue, rather I know exactly where it's bombing out. However, I couldn't understand why the SensorBaseClient would timeout after my committed SQL transactions. The transaction that seems to be causing the problem resides in deleting Hackystat-Devcathlon test users defined in the unit test. It might have something to do with deleting a non-existing row and returning an unhandled Warning from Derby. Anyhow, I'll try to dig something up from the db log files and hopefully resolve this situation soon.
Code:
All source code changes for this milestone can be found, here.
The code distribution for this implementation can be retrieved with SVN. This distribution is a branch of the current trunk distribution, which will be merged onto later.
Finish building database schema and get started on UI improvements. Here are some thoughts on UI improvements:
- Search query for User-profiles, teams and matches (browse pages)
- Add a 'challenge' button for each opposing team profile page. This button will trigger an appendTeam method for appending teams for a new match. The user then navigates over to the Match page to see 'pending' match creations with all teams selected from visiting each team profile page. A user is limited to one pending list per session, and will be able to either accept the new pending match creation, or discard it for an entirely new form.
In my last posting, I mentioned about ditching BDB for Apache Derby DB as Devcathlon's database back-end. I'm glad to say that that was a reasonable choice and most likely a better one in the long run. For this entry, I'll mention about my progress with the application and what I have in store for the next few weeks.
Starting out...
At first I decided to create the database using traditional SQL dialects for building the tables necessary for Devcathon. Below is the necessary Derby configurations and the resulting db schema.
Derby configuration:
Installed and included an environment variable for Derby.
Created DevcathlonDerbyClient.java class (within the same directory of Devcathlon's Jetty-server Start class). The class includes the connection details and initialization of Devcathlon's database. It was implemented using Derby's included JDBC client driver, ClientDriver class. Next I included the client driver for the start of Devcathlon's application in Start.java. Then I started Derby's Network Server, (ran startNetworkServer[sh|bat]) on port 1527. The application fired fine, and Devcathlon was able to communicate successfully with its database.
SQL DB Schema 'Devcathlon':
The full schema file can be found, here. This includes my quick implementation of Devcathlon's database schema for the following tables: Profile, Team, Project, Match and Event.
For convenience, the file is read in by the DevcathlonDerbyClient driver, then parsed and finally executed to initialize the database.
After spending some time reading up on ORMs, I decided to give it a try. I last mentioned about ORMs in my last few postings; this week was the real test and eventual implementation of an ORM, called Hibernate. I had a few other choices for a JPA + EJB standards (OpenJPA), but I think that Hibernate makes for an advanced extension to these APIs. Considering the immense support (online tutorial/manual) and popularity (in forums), I was able to get up and running with Hibernate within less than an hour.
Hibernate-Devcathlon configuration:
I downloaded the distribution (v3.3.1) and extracted it into to my local namespace. I also included an environment variable that links to the root directory of Hibernate, called HIBERNATE_HOME. Next, I modified the Ant build.xml to include Hibernate's required jars for compilation. Then I created a Hibernate configuration file, called hibernate.cfg.xml. The configuration file defines the SessionFactory settings, which includes the database connection(s), hibernate.cfg.xml files, pool connectivity, and debug options. Below is one possible rendering of an hibernate.cfg.xml file for Devcathlon. *Note that Devcathlon will default to looking for this configuration file under the ~/.hackystat/devcathlon/db/hibernate/hibernate.cfg.xml. This is intended so that system administrators can define their own configurations that may require a different database, or whatever is most optimal to their settings.
<hibernate-configuration>
<session-factory>
<property name="hibernate.connection.driver_class">org.apache.derby.jdbc.ClientDriver</property>
<property name="hibernate.connection.url">jdbc:derby://localhost:1527/devcathlonDB;create=true</property>
<property name="hibernate.default_schema">Devcathlon</property>
<property name="connection.pool_size">1</property>
<!-- SQL dialect -->
<property name="hibernate.dialect">org.hibernate.dialect.DerbyDialect</property>
<property name="current_session_context_class">thread</property>
<property name="cache.provider_class">org.hibernate.cache.NoCacheProvider</property>
<property name="show_sql">true</property>
<property name="hbm2ddl.auto">create-drop</property>
</session-factory>
</hibernate-configuration>
Loading this configuration file, required a basic HibernateUtil.java class to handle creating thread-safe, session-per-request SessionFactory. This class is intended to handle initializing one global session, providing pool connections for less overhead on the database, and closing a session. Most importantly, this will be the main communication between our data objects and Devcathlon's database. *Note on property, "hbm2ddl.auto" will be switched to a value of "create-drop" for testing purposes only. On an actual production server, this configuration file should just include a "create" value (without the quotes) for permanence.
Since I started out with building the db schema manually, it really benefited in creating the relational mappings for Hibernate. I could have gone with Hibernate's bottom-up approach by reverse engineering the process to get generated (Java annotated) mappings, but I wanted to see how close my schema was compared to the Hibernate's interpretation of Devcathlon's related classes, and mainly since all the Java persistent classes (ordinary POJOs) have already been built. Thus this is more of a top-down strategy that requires developers to build these relational mappings first, and finally have Hibernate auto-generate and apply its interpreted database schema.
Structural pattern to defining and applying relations:
Define the relational mapping of a persistent class entity, ex: User, Profile, Team, etc.
Add these relational mappings (*.hbm.xml) to the session configuration, by simply adding them programmatically in HibernateUtil.java.
All *.hbm.xml relations-per-entity will reside next to their declared entities and within the same folder.
Creating first relational mappings:
For initial testing purposes, I applied my first relational mappings for the User-Profile class models. The two class models have a one-to-one relation, or simply explained: A User has-one Profile and a Profile belongs-to a User. For a technical explanation, a Profile model would contain the primary key constraint, which 'references' the User model. The relationship will be represented as a bi-directional relationship, for equal referencing by both models.
Below is one way of creating the relational mappings through Hibernate. I could have also followed JPA standards and included some supportive Annotations, but that would eventually become too obstrusive (adding annotations) and more work (removing annotations) if we ever decide to use another ORM. Plus, using these xml configuration files is simple and easy to interpret.
Defining the user.hbm.xml:
<hibernate-mapping package="org.hackystat.devcathlon.engine.user">
<class name="User" table="ACCOUNT">
<id name="id" column="USER_ID">
<generator class="native"/>
</id>
<property name="username" type="string">
<column name="USERNAME"
length="16"
not-null="true"
unique="true"/>
</property>
<property name="email" type="string">
<column name="EMAIL"
length="20"
not-null="true"
unique="true"/>
</property>
<one-to-one name="profile"/>
<property name="password" type="string"/>
<property name="firstName" type="string"/>
<property name="lastName" type="string"/>
</class>
</hibernate-mapping>
Defining the profile.hbm.xml:
<hibernate-mapping package="org.hackystat.devcathlon.engine.profile">
<class name="Profile" table="PROFILE">
<id name="id" column="USER_ID">
<generator class="foreign">
<param name="property">user
</generator>
</id>
<one-to-one name="user" constrained="true"/>
<set name="emailAddresses" table="PROFILE_EMAIL_ADDR">
<key column="PROFILE_ID"/>
<element type="string" column="EMAIL_ADDR"/>
</set>
<property name="screenName" type="string"/>
<property name="firstName" type="string"/>
<property name="lastName" type="string"/>
<property name="bioInfo" type="text"/>
<property name="gender" type="string"/>
<property name="avatar" type="string"/>
</class>
</hibernate-mapping>
Applying the db store:
Since Devcathlon's folder architecture is quite organized, and well thought out for future implementations of a database in mind. It was almost an effortless job on my part, considering that all I needed to do was send a create, read/load, update or delete to Derby's database. Each CRUD function was beautifully built and implemented within each entity's engine manager. So rather than storing everything in the Devcathlon session store, everything gets applied to Hibernate's SessionFactory. Below is a usage template for applying this SessionFactory:
[User | Profile | Team | Match]Manager class:Running JUnit tests.
public function foo() {
...
Session session = HibernateUtil.getSessionFactory();
session.beginTransaction();
// regular java program logic, here
// create the query to communicate with our database.
session.createQuery("..."); //DML statements
session.getTransaction().commit();
...
}
I ran into some issues recently after running JUnit tests with my implementation, caused a SensorBaseClient timeout 1001 error. The error only occurs while running 'testNoData' for the very first test Event implementation (TestBestCoverage). It's not an intermittent issue, rather I know exactly where it's bombing out. However, I couldn't understand why the SensorBaseClient would timeout after my committed SQL transactions. The transaction that seems to be causing the problem resides in deleting Hackystat-Devcathlon test users defined in the unit test. It might have something to do with deleting a non-existing row and returning an unhandled Warning from Derby. Anyhow, I'll try to dig something up from the db log files and hopefully resolve this situation soon.
Code:
All source code changes for this milestone can be found, here.
The code distribution for this implementation can be retrieved with SVN. This distribution is a branch of the current trunk distribution, which will be merged onto later.
svn checkout \Plans...
https://hackystat-ui-devcathlon.googlecode.com/svn/branches/devcathlon-derbydb \
hackystat-ui-devcathlon-derbydb --username <your-user-name>
orsvn checkout \
http://hackystat-ui-devcathlon.googlecode.com/svn/branches/devcathlon-derbydb \
hackystat-ui-devcathlon-derbydb
Finish building database schema and get started on UI improvements. Here are some thoughts on UI improvements:
- Search query for User-profiles, teams and matches (browse pages)
- Add a 'challenge' button for each opposing team profile page. This button will trigger an appendTeam method for appending teams for a new match. The user then navigates over to the Match page to see 'pending' match creations with all teams selected from visiting each team profile page. A user is limited to one pending list per session, and will be able to either accept the new pending match creation, or discard it for an entirely new form.
Sunday, June 7, 2009
[GSoC09] Further DB Investigations and Vision updates
Purpose
This entry will be a brief progress report about what I've done thus far. Currently, I have no new discoveries except for coming to terms with using some kind of database. I've also put out a wiki document for Devcathlon's vision, here (in progress).
Thinking about what's best...
Durg the past week I did some further vestigations and decided to revisit Derby's RDBMS. A comment from my last postg, by Aust Ito made me realize the potential flaw to usg a non-SQL embedded database. Considerg the long-term effects of usg a non-SQL DB would likely attribute to less flexibility and portability of a DB system. Especially if future developers of Devcathlon decide on portg its DB to another enterprise DB, this could be a very pastakg and undocumented process. Austin also mentioned about using an ORM, and I might also consider implementing Hibernate. I've been doing a lot of investigative work (mostly reading) on the topic of ORM's and so far I've noted on some benefits. By definition an object/relational mapping supports full object modeling, including the composition, inheritance, polymorphism and peristence. ORM's, like Hibernate are totally transparent (persistent layer), since ordinary classes do not require any special base class or interface implementations. Its also vendor independent, because it abstracts the underlying SQL database and similar SQL dialects out of the application. Although capabilities of a database may differ between systems, you can expect building a more cross-platform application using ORM.
Plans for this week...
Finish Vision document to include technical review and goals for Devcathlon v2.0.
Continue implementing Derby DB and read up on Hibernate. Hibernate and ORM's itself is still a new topic for me, and I'll be interested to hear about any other comments that attribute to using this persistence framework with Derby DB.
This entry will be a brief progress report about what I've done thus far. Currently, I have no new discoveries except for coming to terms with using some kind of database. I've also put out a wiki document for Devcathlon's vision, here (in progress).
Thinking about what's best...
Durg the past week I did some further vestigations and decided to revisit Derby's RDBMS. A comment from my last postg, by Aust Ito made me realize the potential flaw to usg a non-SQL embedded database. Considerg the long-term effects of usg a non-SQL DB would likely attribute to less flexibility and portability of a DB system. Especially if future developers of Devcathlon decide on portg its DB to another enterprise DB, this could be a very pastakg and undocumented process. Austin also mentioned about using an ORM, and I might also consider implementing Hibernate. I've been doing a lot of investigative work (mostly reading) on the topic of ORM's and so far I've noted on some benefits. By definition an object/relational mapping supports full object modeling, including the composition, inheritance, polymorphism and peristence. ORM's, like Hibernate are totally transparent (persistent layer), since ordinary classes do not require any special base class or interface implementations. Its also vendor independent, because it abstracts the underlying SQL database and similar SQL dialects out of the application. Although capabilities of a database may differ between systems, you can expect building a more cross-platform application using ORM.
Plans for this week...
Finish Vision document to include technical review and goals for Devcathlon v2.0.
Continue implementing Derby DB and read up on Hibernate. Hibernate and ORM's itself is still a new topic for me, and I'll be interested to hear about any other comments that attribute to using this persistence framework with Derby DB.
Subscribe to:
Posts (Atom)