Orbitz - Flying at the Speed of Git

  • Overview
    • The proespect of switching to git does not need to seem like a deep chasm
    • in reality, it’s a bit like skiiing. There are obstacles to avoid, but mostly it’s just FAST (Dave would appreciate this)
    • Migration of 600 repos at Orbitz only took 3 months
  • History
    • 2000–2006: CVS
    • 2006–2012: Commercial VCS with great branching, 597 Repos
    • 2012+: Git
    • Not using git is slowing us down!
    • Every time we looked at a new tool, we faced the fact that it would take 100s of hours of work to integrate with the current VCS
    • Missing out on a lot.
  • Overcoming Inertia
    • Progressive teams were running git on their own servers
    • Git Champions guding the process from start to finish
    • Internal buzz for git was high (90% of dev team in favor!)
  • Where do you start?
    • Github:enterprise -> Github you host
    • Gitorious -> OSS you install yourself, but you can buy support
    • They picked Gitorious, mostly because of strict hardware requirements
    • In the middle of their migration, Stash was announced. (jerks!)
  • Importing history
    • Leverage git fast-import for transferring VCS history
    • You can either take only the current branch (this is what the linux kernel did) or use one of the many tools to migrate
  • How we switched
    • Champions built new tols and were early adopters
    • Existing VCS -> Release Scripts -> Artifactory
    • Git -> Jenkins -> (Same) Artifactory
  • Tools in Orbitz Build Chain
    • Proprietary CLI/Proprietary GUI + Custom Ruby Scripts + Apache/custom rails app
    • CLu, IDEs, SourceTree, More! + Jenkins + Artifactory
  • We held training, not on Git, but on Git at Orbitz
    • What the new publishing path would look like
    • What kind of branching mdoel we would use
  • Approach
    • Repo-by-Repo basis
    • Tools + Docs = Good Luck!
    • 30 minutes, most of the time, for small projects
    • There was a group out of Sydney whose tool took greater than 24 hours to migrate, across the world
  • Proprietary System
    • Over 100Gb in 597 repos
    • Chose to migrate 329. 180 were closed and 88 were deprecated
    • Boiled down to only 20 GB in git!
    • Old system left around for up to 5 years just in case
    • each repo set to r/o after migration
  • Timeline
    • Sep 2011 - tool eval
    • Mar 2012 - building tools/testing
    • Jun 2012 - migrations in full sitng, several teams every week
    • aug 2012 - final repos migrated, ols system r/o
    • 11 mos from start to finish
    • Stash was announced in May, right before final migration
      • Decided they wanted it, but as a phase 2.
  • Costs
    • 2500 man-hours across the company!!
    • Decreased license costs paid for this in less than a years (damn)
  • Immediate benefits
    • 20% faster builds across the board
    • Remote teams finally felt like normal/first-class customers
    • Self-service
  • Why do we love git?
    • Distributed - equal performance regardless of locatoin
    • Cheaper, faster, less administrative overhead
    • No more custom integration!!
  • What could we have done better
    • NFS and hard links do NOT get along
      • git uses hard links when you create a fork
      • ~400 repositories at 20 GB turned into ~3000 developer forks at 80 GB
    • Stash does 2 things to avoid this problem
      • pull requests between branches
      • uses git alternates to share object space between repos so that there’s no duplication.
        • means there are some other things you have to worry about, but Stash handles them behind the scenes
        • totally FS and OS agnostic
  • Hindsight is 20/20
    • Locking and moving repos one at a time was easy
    • Same process for mega repos and some of our tools were tightly coupled to our vcs, which was hard
    • Migrating history was useful for some, but not as a standard process
  • In the last year
    • Migrating to Stash because of JIRA
    • Stash Hooks
      • Automated SOX controls
      • Automated deployments
      • 3–4 other teams who have already built stash hooks because they love it so much
Oct 2nd, 2013

Comments