The Blog

Posts from December 2012

Dec 28

Game Face

By David Czarnecki

“Game Face” will be our weekly round-up of our internal and external open source work here at Agora Games. Internal open source refers to our public projects that you can find over at our Agora Games GitHub account. External open source work refers to projects that we contribute to in off-hours and may or may not have anything to do with video games because we’re swell folks like that. Pretty simple right? Here goes…

leaderboard-coffeescript

leaderboard-coffeescript is our leaderboard project, originally written in Ruby, ported to CoffeeScript. Both this version and the leaderboard-python port have 100% feature parity with the original Ruby library. We will do a formal 1.0.0 release of this library once we work out what to do with respect to naming since there is an existing leaderboard library available from the Node Package Manager.

Contributor(s): David Czarnecki (GitHub, Twitter)

 

Dec 21

Game Face

By David Czarnecki

“Game Face” will be our weekly round-up of our internal and external open source work here at Agora Games. Internal open source refers to our public projects that you can find over at our Agora Games GitHub account. External open source work refers to projects that we contribute to in off-hours and may or may not have anything to do with video games because we’re swell folks like that. Pretty simple right? Here goes…

gamertag

gamertag is an interface for accessing your XBOX Live profile and game data. In the 1.0.3 release, we removed the version numbers in the dependencies and development dependencies for the gem. We also merged a pull request to escape user input when creating an URI.

Contributor(s): David Czarnecki (GitHub, Twitter) and Pedro Nascimento (GitHub)

leaderboard

leaderboard allows you to build leaderboards using Redis. We released version 3.0.1 this week to fix a bug in remove_member that would remove all of the optional member data.

Contributor(s): David Czarnecki (GitHub, Twitter)

leaderboard-python

leaderboard-python is our leaderboard project, originally written in Ruby, ported to Python. We released 2.2.1 that updates the remove_member method to also remove the optional member data for the member being removed.

Contributor(s): David Czarnecki (GitHub, Twitter)

Dec 14

Game Face

By David Czarnecki

“Game Face” will be our weekly round-up of our internal and external open source work here at Agora Games. Internal open source refers to our public projects that you can find over at our Agora Games GitHub account. External open source work refers to projects that we contribute to in off-hours and may or may not have anything to do with video games because we’re swell folks like that. Pretty simple right? Here goes…

In 2011, we open sourced 22 projects. In 2012, we open sourced 33 projects. We’re looking forward to 2013! You can find all of our open source projects from us on the Agora Games GitHub organization.

Here’s a look back at all the 33 projects that we open sourced in 2012 (in alphabetical order):

  • action_pusher - Render views to Pusher from anywhere in your application. Pusher users often want to transmit JSON or HTML to the browser upon model events, but models have no business generating either. ActionPusher allows you to render data to Pusher using your existing view templates from an observer or model.
  • amico - Relationships (e.g. friendships) backed by Redis.
  • beta - Beta is an access control library that uses a Redis whitelist to control authorization. It relies upon the Rails.env variable to check against its list of environments to trigger on.
  • blackjack - The classic card game written using Gosu.
  • bnet_scraper - BnetScraper is a Nokogiri-based scraper of Battle.net profile information. Currently this only includes Starcraft2.
  • bracketeer - Bracketeer is a BracketTree template creation tool that uses d3js to visually create new bracket templates.
  • bracket_tree - BracketTree is a bracketing system built around the BracketTree Data Specification, which uses a three-section data structure built on top of JSON to convey the visual representation, progression logic, and seed mapping in a serializable format. For more information on the data specification, please read the BracketTree Data Specification.
  • bracket_tree.renderer - BracketTree.Renderer is a d3-based bracket rendering system for all BracketTree tournaments. It is designed to interpret the BracketTree data visually based on the Renderer selected. There are a number of renderers built-in, as well as the ability to create your own.
  • brewscribe - Brewscribe is a Beersmith2 (.bsmx) file parser.
  • coffee_bean - A ruby gem for kickstarting a CoffeeScript project.
  • darksky - Ruby gem for retrieving data from the Dark Sky API. The Dark Sky API lets you query for short-term precipitation forecast data at geographical points inside the United States.
  • gamercard - Retrieves and parses an Xbox Live Gamercard for a player, providing a hash of the relevant data about the player or the raw HTML.
  • gem_repackager - Have you ever been without internet and needed a gem but it’s in the wrong RVM gemset? How about attempting to correct a problem with a production environment and need to clone the exact gems available? Perhaps you are attempting a full app stack backup for compliance purposes? Gem::Repackager packages one or more of your installed gems back into .gem files for easy transportation. Gem::Repackager comes with a command-line utility to facilitate, with easy extensibility in the code as well.
  • GWFSelect-for-jQuery-UI - Based very loosely on Tom Moor’s plugin for selecting Google’s hosted fonts from a dynamically-generated drop-down list. This version uses jQuery UI’s widget factory to provide the standard tools for accessing and manipulating the plugin’s state programmatically.
  • kairos - Kairos provides time series storage using a Redis backend. Kairos is intended to replace RRD in situations where the scale of Redis is required, with as few dependencies on other packages as possible. It should work with gevent out of the box.
  • leaderboard_factory - Helpful tools for defining a bunch of leaderboards associated with your objects. Builds on the leaderboard gem.
  • node-amico - This is a NodeJS port of amico using CoffeeScript.
  • node-darksky - A node.js module for integrating with the Dark Sky API.
  • oembedr - Lightweight, Flexible OEmbed Consumer Library.
  • php-bracket_tree - BracketTree is a Tree-based bracketing system designed to aid in the creation and representation of tournament brackets. It uses a binary tree under the hood to maintain the bracket, and the BracketTree Data Specification to control progression and standardize representation. This is a port of the original Ruby client.
  • prometheus - Prometheus is a lightweight, modular framework built on Thor to quickly create beautiful command-line interfaces for your gems. It provides a standardized layout with generators, smart configuration, and an interactive console to work with your tasks.
  • punchr - Ruby gem for interacting with the Punchfork API.
  • rduration - Simple utility for parsing durations from strings and comparing them. Basic math is also supported.
  • redis_pagination - Simple pagination for Redis lists and sorted sets.
  • seed_list - Seed management for tournament brackets.
  • silver_spoon - Entitlements in Redis. A simple semantic wrapper around Redis hashes for adding, removing, retrieving and checking existence of entitlements.
  • snake_skin - Python package skeleton tool, similar to the bundle gem command in Bundler.
  • spinel - Spinel is a new free and open source game engine under development that utilizes mruby as its integral scripting layer. Under the hood is C/C++, however wherever possible the engine uses Ruby.
  • streak - streak is a gem for calculating win/loss streaks. It uses Redis as its backend for collecting the data.
  • streak-coffeescript - streak is a library for calculating win/loss streaks. It uses Redis as its backend for collecting the data. This is a CoffeeScript port of the original streak Ruby gem.
  • streak-python - streak is a library for calculating win/loss streaks. It uses Redis as its backend for collecting the data. This is a Python port of the original streak Ruby gem.
  • strumbar - Strumbar is a wrapper around ActiveSupport::Notifications with preconfigurations for basic instrumentation to be sent to statsd.
  • windbag - Windbag is an event notification system for Rails. Using Windbag with model observers, you can notify users about changes they’re interested in over a variety of transports (WebSockets, email, SMS, Twitter, etc).

 

Breakdown by language: Ruby (22), Python (3), CoffeeScript (3), JavaScript (2), C++ (2), PHP (1)

Open source contributors from our engineers (in alphabetical order): - David Czarnecki (GitHub, Twitter) - Logan Koester (GitHub, Twitter) - Jack Letourneau (GitHub, Twitter) - Andrew Nordman (GitHub, Twitter) - Tom Quackenbush (GitHub, Twitter) - Aaron Westendorf (GitHub, Twitter) - Matthew Wilson (GitHub, Twitter)

 

Thank you to all the external contributors to our various projects (in alphabetical order): - Jason Baker (GitHub) - Skip Baney (GitHub) - Greg Banks (GitHub) - Jon Barber (GitHub) - Bradley Buda (GitHub) - Jesse Cooke (GitHub, Twitter) - Steven Davidovitz (GitHub, Twitter) - John Gadbois (GitHub, Twitter) - Paul Gallagher (GitHub) - Kate Gengler (GitHub) - Joey Imbasciano (GitHub) - Seb Jacobs (GitHub) - keysolutions (GitHub) - Vitaly Krugl (GitHub) - Jeffrey Lee (GitHub) - Szymon Nowak (GitHub) - Rodrigo Tassinari de Oliveira (GitHub) - Chris Roby ( Twitter) - Marian Rudzynski (GitHub) - Stephen Sugden (GitHub) - zombor (GitHub)

Dec 7

Game Face

By David Czarnecki

“Game Face” will be our weekly round-up of our internal and external open source work here at Agora Games. Internal open source refers to our public projects that you can find over at our Agora Games GitHub account. External open source work refers to projects that we contribute to in off-hours and may or may not have anything to do with video games because we’re swell folks like that. Pretty simple right? Here goes…

haigha

haigha is our simple to use client library for interacting with AMQP brokers. We released 0.5.9 this week with support for array lists in headers.

Contributor(s): Aaron Westendorf (GitHub, Twitter) and Joey Imbasciano (GitHub)

hipchat-api

hipchat-api is a Ruby gem for interacting with the HipChat API. In the 1.0.5 release, support was added for the roomstopic and usersundelete API methods.

Contributor(s): David Czarnecki (GitHub, Twitter)

leaderboard

leaderboard allows you to build leaderboards using Redis. We released version 3.0.0 this week which incorporates the following changes:

  • Added rankmemberif and rankmemberif_in methods that allow you to rank a member in the leaderboard based on execution of a lambda.
  • No longer cast scores to a floating point automatically. If requesting a score for an unknown member in the leaderboard, return nil . Under the old behavior, a nil score gets returned as 0.0. This is misleading as 0.0 is a valid score.

  • Removes :use_zero_index_for_rank_option as valid option for requesting data from the leaderboard. Original proposal

  • Optional member data is stored in a single hash. Original proposal

  • Adds :sort_by as valid option for requesting data from the leaderboard. Original proposal

  • Removes :with_scores and :with_ranks as valid options for requesting data from the leaderboard

 

A version jump to 3.0.0 was required since the storage of optional member data for a leaderboard was backwards incompatible with the 2.x release.

Contributor(s): David Czarnecki (GitHub, Twitter)

leaderboard-python

leaderboard-python is our leaderboard project, originally written in Ruby, ported to Python. We released version 2.2.0 this week, which has functional parity with the Ruby library. You would be able to use either library to interact with leaderboard data in Redis. Going forward, we will try as best we can to always maintain functional parity between the Ruby and Python implementations.

Contributor(s): David Czarnecki (GitHub, Twitter)

stache

stache is our Rails 3.x compatible Mustache/Handlebars Template Handler, with support for partials and a couple extra niceties to make sharing the raw templates with client-side javascript a little easier. We integrated a pull request to be able to use template files without format in extension for mustache.

Contributor(s): Matthew Wilson (GitHub, Twitter) and Kate Gengler (GitHub)

Dec 3

Benchmarking Redis Transactions

By David Czarnecki

I recently released version 3.0.0 of our leaderboard library. The library is a semantic wrapper around Redis sorted sets to store and interact with the leaderboard data. When I first released the library, I had put a section in the README on Performance Metrics. This provided a useful benchmark from which to gauge performance regressions that might be introduced when adding new functionality.

I re-ran the benchmarks on my MacBook Pro, 2.5 GHz Intel Core i7, 8 GB 1333 MHz DDR3, Mountain Lion 10.8.2, Ruby 1.9.3-p327, Redis 2.6.7, redis gem 3.0.2, Redis persistence disabled.

By default, the code will use a Redis transaction to rank a member in the leaderboard. This is to accommodate being able to optionally store data alongside the leaderboard. However, in hindsight, the common case may just be ranking (or updating) a  member in the leaderboard without this data.

Time to insert 10 million records w/ Redis transactions per insert: 23 minutes

It seems like Redis transactions, where they’re really not needed, is having a performance impact. I removed the transaction parts of the rank_member method in the library temporarily and re-ran the benchmark code.

Time to insert 10 million records w/o Redis transactions per insert: 13 minutes

This number makes more sense to me and is inline with the original benchmark time of 15 minutes accounting for faster hardware, updated Redis version and updated Redis client library. Awhile back, however, I specifically added a few methods to do bulk insert of member data in a leaderboard. I modified and re-ran the benchmark code to rank members in the leaderboard, 1 million per transaction.

Time to insert 10 million records w/ Redis transaction inserting 1 million records at a time: 7 minutes

When I initially re-ran the benchmark against the leaderboard 3.0.0 code, it felt slower, but I needed actual numbers to backup that feeling. Now I can benchmark the code again if I decide to add in a conditional to not use a Redis transaction in the rank_member method if optional data is not supplied.