Oisín Hurley's Weblog

old dog, new tricks

Archive for the ‘open source’ Category

Bog Body: Committing to Open Source

with one comment

Backstory: I’m spending some time trying to find 10 spare gigs on my laptop to house the monster Xcode 4 install. During the cleanup, I’ve unearthed several dozen bog bodies from my previous software and employment versions. Some are embarrassing, but they are like eccentric relatives, I cannot bring myself to deny them, plus I enjoy seeing the effect they have on other people. So, rather than throwing them out, I thought I might share them here so that we can all have a laugh.

This one was from when I worked in an Open Source organization embedded in a Closed Source organization – when OSS suddenly became trendy in Enterprise Software. It was a draft of a primer of how to do Open Source When You Have No Idea How It Works, meaning how to take part in projects and run a stable of OSS developers successfully. I don’t think it got beyond a second draft. I still think OSS is a better way to do most things, however it does have its own pathologies, which at least are fairly visible.

Committing to Open Source

So, you want to try out this OSS thing. It’s the new hotness in the press, and your competitors are all about competing only on the 20% and contributing to and using Open Source for 80% of their delivery. You could get fired if you don’t jump on the bandwagon. Measuring your success is crucial. Your old metric of revenue linked to products won’t really work with Open Source. You will need to build a force of highly-competent developers that can consistently deliver value to customers.

OSS is a services engagement in most cases. Customers must be kept happy – you need capable and competent developers and you got to have a plan to keep them coming on-stream. This is vital when you are dealing with potentially the most mobile workforce in the world.

Key Aspects of Open Source (OSS) vs. Secret Source (SSS)

  • SSS is about chain-of-command
  • OSS is about tribal pressures
  • SSS is about running a factory
  • OSS is about being a patron
  • SSS is about being regulated and policed
  • OSS is about self regulation by application of taboo
  • SSS success is about moving up the hierarchy
  • OSS success is about moving out into the world

Patronage in OSS
You are a patron. Think about how that model works – those receiving patronage want to practice their craft. Those giving patronage want to enhance their standing amongst their industry peers. By enhancing their standing the patron can build a reputation and upon that, a dominance.

Those receiving patronage will brook some interference in the execution of their tasks, but not much, so you need to maintain a loosely-coupled connection to the body corporate. This can be difficult. You need someone who has trust from the stable of developers, and also can communicate the corporate messages in a way that makes sense – really this is not about spouting corporate goals, but being able to construct competitive challenges like – “We want to kill XYZ, how do we do it?”, or “That QRPX repository looks good, but we can do better”. Basic, tribal goals, not code-speak for corporates, no mission statements.

OSS developer is a highly mobile position. If you are a good patron, your reputation will spread. You build the reputation for a year or two, then you can start trying to poach names from the other projects that you are up against, poach the bloggers, do this by word-of-mouth. Every single one you persuade will give you PR you can’t manufacture. You are also hitting those projects where it hurts, by taking their staff directly.

How to be a Good Patron

What you do is strengthen each developer’s community immersion. One of the big winners is to increase their physical presence in the OSS milieu. Allow each developer to attend two conferences per year, more if they are speaking. Profile the conferences, and make places available, limit them to encourage submissions. Review your guys’ blogs and suggest topics to them, keep feeding them competitive nuggets. You also need to let them investigate stuff that is new and interesting. Some small-scale and locally-constrained brownian motion is fine, as long as the general trend is moving forward. Make sure you send them to competitors’ conferences, even things like Microsoft conferences. These people are your eyes and ears, developers are mostly honest, and they will give you a level of relevant information you have never had before.

Making Good OSS Developers

Firstly, keep the ones you have. If you are a Good Patron, your attrition should be tiny. However, you need to educate developers to turn them into OSS developers, it is not an instinctive thing. Programs for committership must be established. A mentoring system must be present. Committers must be regularly assessed on their contributions in the areas of Code, Documentation, Bugfixing, Customer Queries, Outreach (blogs/speaking/etc). Developers must have training and committership in more than one project, and may move on to project-centric, or customer engagement-centric majors. Competencies need to be tracked, and gating system put in place for customer engagements, based on using this competency system. This will help keep a strong reputation for technical and product capability in field assignments.

Some of the Roles/Structures You Need

Much of what follows is stating the obvious. There is a strong bias toward the developer satisfaction theme here, since that is the element that is the most different in approach to the SSS method.

  1. The Project Selection Group – a small group that discusses and decides what projects to be involved in, what to pull out from, what projects to start and promote, etc. Meets quarterly and produces auditable decision trail. Revisit decisions on substantive grounds only after a test period to ensure commitment.
  2. The Community Manager – one committer in a project has the role of Community Manager. They have a number of responsibilities. One is as the eyes and ears of the project selection group as to the state of the project, community health, diversity, etc. Another is to help to ensure the smooth running of the project and its delivery to users, so community manager role should be strongly biased towards repeatable builds, good website material, strong demos, blogging about the project, outreach at conferences and the like. Also this person is responsible for gauging developer performance in the project (doc, code, marketing, customers). One per project. All project leads and community manager folks must blog about the project and aggregate any blogs that regularly mention it. Everyone needs to read these.
  3. The Doc Team – if one thing adds value to OSS, it’s the documentation. The Doc Team gets close with the community manager to see where the docs are missing, what needs to be done, etc. An enhancement to documentation on the OSS project website is a big plus to the project, but excellent comprehensive documentation is something that people will pay for.
  4. The Tools team – another value add is pain reduction. Hard-core developers will have no issues working directly with the OSS project elements as downloaded from the website. The great majority of developers will need tools. A dedicated team is required to produce tools and shortcuts to help customers in certain places to increase productivity. Act on feedback from the services guys, other field reps and the community.
  5. The Distro team – there’s a group of individuals that are responsible for keeping the distributions ticking over and building – they are responsible for nightlies, weeklies, integration builds and the eventual distribution. They make the distros.
  6. The Test team – team that builds integration tests that are used as smoke tests for distro quality. They also design the tests.
  7. Web team – don’t have one of these – outsource it, but make sure the developers can update at any time.
  8. Marketing Support – primary marketers in the OSS community are the developers – they are the reputation enhancers. However more advertizing-style marketing is required that is full-time. Competition is rife in these communities and it goes to the deepest level. Every now and then stir the pot with some controversy. People can’t help but respond. Don’t be destructive – keep the bridges uncharred.
  9. Administrative Support – office management, expenses, travel arrangements, etc are all things that developers are not so good at.
  10. Training/Services – competency-based tracking of developers employed to stream capabilities to customers. It is important that every developer takes some time every year to be in front of a customer.
  11. Legal – maximally important when engaging with enterprise customers. Consider paralegel education for developers, or at least support them where feasible. Employ lawyers who have provenance in this area.

Dealing with the Mud-Slinging

When operating (‘competing’) in the OSS arena, attacks from others can be directed towards the software, or to the organization. Challenges to the software are easily met – if they are untrue, show that is the case; if they are true, then address the issue, and make a show that you have addressed it and that your detractor will have to think of something else to complain about. Attacks on the organization are generally harder to grasp.

The top four muddy missiles are usually the following (where for X insert company name):

  • X has not acted in the best interest of the community
  • X has been divisive in the community
  • X has been spreading FUD (fear, uncertainty, doubt)
  • Our software has displaced X in every account we went into

These kinds of accusations are reputation-tarnishers and need to be addressed. Accusers need to explain their assertion, and it needs to be dealt with in the open. They will need to cite examples for any challenge. Be sure to track every single engagement with them at the customer level. Build competencies in competitor product so that you can keep ahead – delegate a team to track and build small systems with the competitor product. Competition needs to be strong and you need to be up to date all the time, and be ready with insight or analysis, indeed produce these regularly.

If you come across failures in competitive products in real-world scenarios, you need to record them and save them for deployment when necessary – for example when you get attacked or when the opponent is in a weak position. Keep references to your own successes.

…cable ends…

Advertisements

Written by oisinhurley

August 26, 2011 at 6:21 pm

Enforce Web API version compatibility with Sinatra filters

leave a comment »

Versioning in APIs is vital if you want to control the lifecycle, rather than it controlling you. If you are using Sinatra to do routing for your Web API, then you can easily stuff all the version compatibility testing into one place – a filter – and not have to propagate it through all your routes. Follow the gist to see the code.

https://gist.github.com/987247

Written by oisinhurley

May 3, 2011 at 12:13 am

Missing in Action from EclipseCon 2011

with one comment

EclipseCon 2011 started today, with an awesome program of events going on over the week. I’ve attended the previous four instances of the conference, and had the signal honour to have been the Program Chair for EclipseCon 2010. Now, alas, I’ve lapsed. Instead of getting a nice easy job after my release from Progress last year, I’ve foolishly decided to startup a software business with a couple of guys. I hope this adequately explains my somatic non-presence at this years showcase of what’s wonderful in the Eclipse world. For example – if I was in California today, I would not have been able to load the mobile part of our product on the phone of a friend-of-a-friend who just happens to be attending a concert with the CEO of a division of a large cellphone company, with instructions to show it to this CEO and get a meeting for us. But if I was in California today, I would be enjoying beers with lots of people that are much smarter and much more dedicated than me.

It’s always a tradeoff.

***

Now that I have a brand new ‘user’ perspective, and am on the outside looking in, there are a few items within the Eclipse Ecosystem that are particularly interesting.

  • EGit – I’m using git exclusively now for source code control, and having good support is very important. I think that the real decision-maker on using git all the time are hosting services like Heroku and Nodester adopting git push workflows for application deployment.
  • DSL-based mobile device project scaffolding – projects like Applause and Applitude, both based on Xtext and mobl, based on Spoofax, can give you a chunk of starting point code to get your mobile application up and running on the cheap. I haven’t taken as much time to study these as I want to – a future blog entry I think.
  • Orion – when I first saw the Bespin project, which then merged with Ace and is now the development-as-a-service Cloud9 IDE, it failed to stir me. However, when I started doing some node.js project experiments, then the option of being able to edit your JS code through the browser suddenly had more appeal, precisely because now you can edit server code. It will be worth a blog entry in its own right at some point (aside – Mr. Orion, @bokowski, just linked to another one, Akshell, a minute ago)

***

I just realized the other day that the last piece of Java/Eclipse a programming I did was in July 2010. Since then I’ve been in this dual world of the mobile app developer – programming
native code on iOS devices, programming Rails 3 and node.js on the server end of things, pushing data into PostGIS and Redis. I didn’t think I would end up here, but it’s been fun so far 🙂 As to the future, I’ve refused to plan anything beyond world domination for the moment. But maybe I can get an Orion talk accepted for EclipseCon Europe

Written by oisinhurley

March 21, 2011 at 11:56 pm

Xcake – Web Services with Sinatra and Heroku

leave a comment »

On Feb 8, I gave a talk at the local Xcake monthly meetup at the Science Gallery in Dublin. These meetings are predominantly loaded with mobile developers, with a worthy smattering of other creative professionals. The talk was on Web services, and how easy it can be to lash something together to test your mobile application development. Many of the mobile dev shops interact with client data through a web service, and making your own local copy with test data can give you a speed advantage in getting your (client’s) app to market. The talk uses Ruby, Sinatra and Heroku as the core tech behind a RESTafarian style of service, with persistence and a public deployment.

The good news is that neither the bucket of tar nor the sack of feathers was required by the audience, there was a distinct lack of torches/pitchforks, et cetera. As a result I’ve posted the slides on Slideshare. They are mostly composed of screengrabs and codedumps, so make sure you view the presenter notes, in which I have expanded on the sparsely populated slides themselves. You can get the sample code on GitHub. I’ll buy beers for anyone who implements simple database authentication and sends me a successful pull request 🙂

Note: I’ve spotted some rendering weirdness with Chrome – if you can’t see the slides, try another browser, or hit the download button.

Written by oisinhurley

February 10, 2011 at 6:49 pm

Posted in Coding, open source, talks, web services

Tagged with , ,

Eclipse ESE 2010 and STP News

leave a comment »

Eclipse ESE 2010 w00t!

Eclipse ESE 2010 starts tomorrow, and I’m going to be there – well, I’ll be there from Wednesday to Friday. I’m delighted, because I had written it off this year, having had to dramatically scale down my Eclipse community involvement. But with the help of Herr Program Chair Bernd, and cheap Eurozone flights, I’ll be helping whip the room into shape in the Build Systems Exposed: Strengths & Weaknesses of Build Technologies at Eclipse panel discussion. Present will be the usual suspects, Henrik “Buck” Lindberg, Jason “Tycho” van Zyl and Nick “Athena” Boldt. Unfortunately, the fourth stooge, Dave “Agile” Carver, is being held incommunicado by his cats and can’t make it. The point of the session is to help you, the attendee Eclipse developer, to get your brain-gear about how to make your projects build, get tested and get packaged. Questions are very important in this regard. Bring ’em.

For myself, I’m too long in the tooth and cynical to insist that there is one true path here, that one of the approaches is the best in all cases. I’ve been in Enterprise Software for the last 17 years, you see. In any case you have to do your own research as to what will work for you – I want you to consider this as school homework. No-one is going to do your assignment for you, because every project is different, and frankly, you need to learn how this stuff works or else you will be doomed next time around. Put it into the schedule.

Maybe that came across a little cranky. I’ve been on both sides of the issue and neither is comfortable, but I think you do have to learn it yourself for the knowledge to stick, so you can maintain it. However, that doesn’t mean I think everyone needs to start from scratch – there’s a base level of mechanics that can be given as an exemplar from which people can extrapolate. Lots of examples exist for this particular sphere of issues, so what I would really like to see would be The Eclipse Ultimate Guide To Building Eclipse By Example, which would take N types of Eclipse project and for each one show the M different ways to build it. A way to kick off would be to take Wayne’s article on building Woolsey with Tycho and do a similar one on building Woolsey with Buckminster (or Athena).

In any case, looking forward to getting there and reminding myself how good German beer can be!

Eclipse STP News

The Eclipse SOA top-level project is now up and running pretty much completely, and with that the merge of the Swordfish and the projects contained by the Eclipse STP top-level project is complete. More than complete, in fact, with the addition of the eBPM, eBAM and Java Workflow Tooling projects, there’s a comprehensive and diverse range of solutions available. Now, there is merely tidying up to do, and the remaining strands of connective tissue (build, web, etc) that kept those projects in place in STP needs to be removed. With this message to the STP and SOA PMCs I’ve initiated that process. Congrats to all projects on their move to Eclipse SOA and best of luck to the combined Tools + Runtime top-level project!

Written by oisinhurley

November 1, 2010 at 11:05 pm

Eclipse Quickie: Changing Buckminster Build To Use Git

leave a comment »

Question: So, you are building a gaggle of projects that use Subversion, and one of them pops up and decides that distributed source code management is now the future and that they have moved to Git. You have a Buckminster build system – what do you do?

Answer: Make a simple change to your RMAP file as follows and just carry on as usual.

Before: we discover the source code of the BPMN Modeler from its Subversion repository.

    <searchPath name="bpmn.plugins">
       <provider readerType="svn" componentTypes="osgi.bundle" mutable="false" source="true">
          <uri format="http://dev.eclipse.org/svnroot/stp/org.eclipse.stp.bpmn-modeler/org.eclipse.stp.bpmn/trunk/{0}">
              <bc:propertyRef key="buckminster.component" />
          </uri>
       </provider>
    </searchPath>

After: we discover the source code of the BPMN Modeler from its Git repository.

    <searchPath name="bpmn.plugins">
       <provider readerType="git" componentTypes="osgi.bundle,eclipse.feature" resolutionFilter="">
          <uri format="{0}/bpmnmodeler,{1}">
              <bc:propertyRef key="workspace.root" />
              <bc:propertyRef key="buckminster.component" />
          </uri>
         <property key="git.remote.uri" value="git://git.eclipse.org/gitroot/bpmnmodeler"/>
         <property key="git.remote.name" value="bpmnmodeler"/>
       </provider>
    </searchPath>

Look for: the readerType has changed to git, there’s a change in the uri format, and we’ve been required to add a couple of new property key-value pairs. Then it works first time (for me at least!). You will need the 3.6 Buckminster stream to get the git readerTypedownload information is here.

I have to say a big thank-you to the Buckminster Team for this piece of software that not only powers the STP build, but also the construction of the complete Helios release!

Written by oisinhurley

June 15, 2010 at 11:02 pm

EclipseCon 2010 Retrospective

with 12 comments

It’s nearly time to return to our scheduled programming, but first a quick retrospective of EclipseCon 2010.

Oisin Rejected My Talk

The danger with writing a retrospective like this is that it can rapidly become a screed of great proportions and hit everyone’s tl;dr button. So I’ll keep this short. What I am reporting here is my own experience plus feedback from both presenter and non-presented folk. I’ll keep the format of the previous articles to focus it.

Exercise

This was a great success this year – better than years when there were half again as many attendees. My only regret was that I missed out on a teeshirt on the first day! Much appreciation to Kim Moir for, er, running with this.

Keynotes

The keynote from Oracle was a lacklustre affair – and watching some of the tweetage that was coming out from their panel later in the day, it’s no surprise since Oracle don’t appear to have decided what they are up to. Especially with the retention of three UIs – Eclipse, JDeveloper and Netbeans. Come budget time, the VP or whoever at the pointy end of controlling those three groups will have to get the hatchet out. Which will fall? JDeveloper is ensconced like a tick – parts of it are in core Oracle products (allegedly). I have to assume that the Netbeans team is larger than the Eclipse team, does that make them more likely to be kept? Of course, it won’t be that simple – perhaps the Netbeans Java tooling people could be transitioned to Eclipse to add more value to the JDT? Anyway, I’m sure we’ll be guessing for a while.

The NASA/JPL keynote from Jeff was a masterful performance – not only the content, which was bang on demographic for a crowd of developers and technophiles, but the structure and production values were excellent. I caught a couple of clips – here’s Jeff talking to David who is supervising the ATHLETE robot in the lab

and here is Jeff’s closing remarks with his Lego buddy Socrates

Robert “Uncle Bob” Martin (Roberto) presented the keynote on software professionalism and it too was excellent work, although on a totally different axis to Jeff’s. This man is a past master of the presentation style, fearless, emotive and ready to challenge his audience. Here’s some short extracts:

  • agile is about destroying the hope that keeps stupid plans alive
  • say no to all forms of bad code
  • say no to dropping your disciplines
  • say no to overtime – know your limits
  • say no to meetings – when the meeting gets boring, leave
  • say no to dumb restrictions on your development process

I think the talk unnerved some people (as evinced by -1’s in the red bucket), so good work Roberto!

Tutorials

I got a goodly amount of positive feedback that a tutorial-per-morning was a nice idea and something that people did look forward to. There was a bit of balancing feedback that some of the tutorials were somewhat disorganized, not fully prepared, had too many slides, didn’t have enough introductory information, or were missing required data. The Program Committee had a meeting on this and we have a couple of proposals to make sure that these kinds of issues won’t occur next year.

Talks

On balance, the 25-minute talks worked, I think. I had a lot of conference-goers saying that they were pleasantly surprised with the shorter format – they got a similar amount of information in a shorter time and could go to more talks. Most presenters stepped up to the plate and did a really good job on condensing their material and cut quickly to demos. Some didn’t and the talks were rushed and hassly. There were a couple of people who appeared to be personally upset about the timing – how on earth could you talk about anything in less than 35 minutes? They got my get over it lecture.

One the other hand, the 12-minute lightnings did not really work at all well this year compared to other years. I think it’s because so many speakers got instructed to cut from standard length to lightning and it was just too difficult to do. I’m thinking we might revisit that for next year.

Feedback was on the high side for good talks, however some donkeys were brought to light too.

Panels

The panels I went to were good. The quality of the moderation was tip-top, the preparation was good, and all were conducted in good humour and with great candour. Controversy can work well in a panel, but it appears that good, solid, experiential data works well too. Big shout out to Dave Carver for his innovative Jeopardy-style approach in moderating the Build and Continuous Integration Panel and use of “the Undead” as a question category 🙂

BoFs and Unconference

This year we seemed to be totally BoF-tastic, with a great amount of activity going on there, but the Unconference got little support. Lesson learned here – the crowd at the ‘Con are more interested in getting together in a social group for listen and learn rather than stepping up and doing their own off-piste talks.

Divers Alarums – being a gallimaufry of tatterdemalion conference qualia
  • e4 Rover Programming Competition – wow, this was a runaway success and frankly will be hard to top. Huge kudos to the NASA guys, Ian, Boris and Ben
  • EclipseCon Tweetup – informal meeting in the bar, always works for me 😉
  • Don‘s sense of humour – “I’m stalling…and you’re all leaving…and now I’ve lost all credibility…”
  • Sergey P’s phones – classic
  • Giving us mugs instead of little rinky-dink cups for coffee – most essential
  • The general buzz around the place since we were in a smaller area was great
Thanks to all…

Well it looks like this turned out to be a screed after all. I hope you enjoyed the conference. I’ll finished with thanks to many people that made the conference work – attendees, thank you for coming; presenters, thank you for presenting your work; keynoters, thank you for crafting your keynotes and delivering them to a tough crowd; program committee, thank you for the hard graft coming up to program announce date; Don, Anne, Gabe, Ian, Lynn, Mary, thank you for hard work in the logistics and outreach departments!

And with that, I leave you all in the safe and capable hands of Chris Aniszczyk, Eclipse Committer, runner, author and Program Chair for EclipseCon 2011.

EclipseCon Stalwarts Quenching Their Thirst

Cheers!

Written by oisinhurley

March 31, 2010 at 9:30 am