TRANSFORM 2022

Last week the Software Underground hosted the latest edition of TRANSFORM, its annual festival of digital subsurface stuff. This year we had a packed schedule, including:

• Folks sprinting on various software projects at the weekend.

• 13 live hands-on tutorials on a wide range of topics.

• Birds of a feather meet-ups to discuss the future of scientific virtual meetings.

• Eight lightning talks on topics from sealice to web apps to scholarly communication.

• The Annual General Meeting of the Software Underground organization.

All this was stitched together by the Slack chat, as usual. You can catch up on what you missed in the #t22-general channel. Our Slack is free to join and open to anyone.

---

The tutorials

Here’s a list of the tutorials, which were very high quality this year. All of them were 90 minutes to 2 hours long, and all are supported by fully open-source software and open-access materials. How awesome is that?

• Getting started in Python, by Robert Leckenby (Agile).

• GSTools, a toolbox for geostatistical modeling in Python, by Sebastian Müller (UFZ, Leipzig).

• What is Markov chain Monte Carlo? By Miguel de la Varga (Terranigma).

• Mandyoc, a finite element simulator for the mantle, by Agustina Pesce et al.

• Machine learning models for geoscience, by Tom Ostersen (Datarock).

• From Zero to Devito (geophysical modeling & inversion), by Rhodri Nelson (Devito).

• Publishing your first Python package, by Matt Hall (Agile).

• PyGIMLi (geophysical modeling & inversion), by Florian Wagner et al. (RWTH).

• OccamyPy, an object-oriented optimization framework for large-scale inverse problems, by Francesco Picetti & Ettore Biondi (Stanford).

• A geophysical tour of mid-ocean ridges, by Leonardo Uieda (U of Liverpool).

• Self-supervised noise suppression, by Claire Birnie & Sixiu Liu (KAUST).

• Julia for geoscience, by Francis Yin (Georgia Tech).

• PyLops (geophysical modeling & inversion), by Matteo Ravasi et al. (KAUST).

If any of those sound good to you, and I hope they all do, check them out in this YouTube Playlist.

---

The lightning talks

As always, the lightning talks were a lot of fun. Check out the Description in this video for the full list of chapters:

---

That’s it for 2022. Hope to see you at TRANSFORM next year!

Transform 2021 rolled out

On Friday we wrapped up the 2021 edition of TRANSFORM, the Software Underground’s annual virtual conference. We stuffed a hackathon, 21 tutorials, 20 lightning talks, and an annual general meeting into a week-long celebration of open subsurface code and data.

Many thanks to our sponsors — especially Studio X in Austin, Texas — and to all of the participants who donated to the conference this year. As a pay-what-you-like event, we depend on generosity to fund the things we do. In return, we are trying to bring some new superpowers to the community. So far, so good.

Just like last year, the event kicked off with a hackathon, which again lasted all week. The event saw 9 projects being worked on. this number included long-lived projects like Subsurface, GemGIS, SEGY-SAK and Striplogm as well as new projects looking at seismic footprint removal and well correlation, among other things. We’ll tell you all about these projects in the coming days.

The heart of the conference week itself was the 21 amazing tutorials — 33% more than we had in 2020! The instructors and presenters this year participated from all over the world:

The 29 instructors and 20 presenters this year came from all over the world.

We’re so fortunate to have scientists in our community that not only spend hours (years!) writing open source software for others to use, but then will also contribute open content to help others use it. Many, many thanks to our team of instructors:

Day 1: Ashley Russell (NOR), Claire Veillard (NOR), Florian Wagner et al (DEU), Anne Estoppey (NOR), Miguel de la Varga (DEU), Diana Acero-Allard (USA), Maria Cecilia Bravo (NOR), Michael Pyrcz (USA), Thomas Martin (USA).

Day 2: Tony Hallam (GBR), Lachlan Grose & Mark Jessop (AUS), Olawale Ibrahim (NGA), Øystein Klemetsdal (NOR), Nathaniel Jones (USA), Bane Sullivan (USA), Seogi Kang (USA).

Day 3: Jørgen Kvalsvik (NOR), Graeme Mackenzie (NOR), Edward Caunt (GBR), Steve Purves (ESP), Matteo Ravasi (SAU), Dewey Dunnington (CAN), Santi Soler (ARG), Irene Wallis (NZL), Katie McLean (NZL).

It wasn’t all Python either — one tutorial used R, another MATLAB/Octave — and we even had a tutorial in Spanish this year. We also had participants from well outside the usual petroleum-rich industries: geothermal, mining, hydrology, and technology were all represented. I hope we continue to see increased diversity in languages, industries, location, and in all dimensions.

The best thing of all, especially if you weren’t able to take part last week? Everything is on YouTube, so check it out, follow along, and learn some new tricks.

One of the really remarkable things this year was the prevalence of tutorials and hackathon projects that combined multiple software projects from the open subsurface stack (which was the goal of TRANSFORM all along). Several events combined two or more of GemPy, Devito, PyVista, Subsurface, segyio/SEGYSAK, or lasio/welly/striplog. It’s exciting and encouraging to see, and signals a new level of maturity among these tools. The future for the open subsurface stack looks bright.

Thank you to the organizing team — Dieter Werthmuller, Ashley Russell, Irene Wallis, Brendon Hall, Rob Leckenby and Sofiyah Mokhtar — for helping pull everything together.

And thank you to everyone that participated in TRANSFORM 2021. I hope you learned something new, and met someone interesting. Please tell your friends and come back next year!

News from the Underground

What’s new this week? This is what…

Find work fast — Earth science is undoubtedly experiencing something of an employment crisis at the moment. Happily, the #jobs channel has been busy lately, with nine positions posted in the last seven days. Three of them at the Norwegian Geotechnical Institute.

Virtually no rain… or pubs — A question about virtual geological field trips highlighted some good resources, including this excellent resource — and map, below — from the MicroMyEarth website, which is run by Arthur Adams, a Canadian geochemist based in Switzerland. If you teach earth science, you should definitely check it out!

Spot the difference — The #python channel is probably the most active channel on the Software Underground Slack, and often sees great debate. This biggest thread this week was in reponse to a question about the difference between Jupyter Notebook and Jupyter Lab, but the answers go well off piste. Check it out.

Arrrrrr, treasure — Although Python wins the popularity contest, there are a great many earth science libraries for the R programming language. Calgary-based geoscientist Danny Coutts took the trouble to find them all, and told everyone about it in the #r-stats channel.

📈 Interactive plots — everyone loves widgets! Later today, at 1600 UTC, Wesley Banfield and Steve Purves will be looking at ways to sprinkle a little bit of JavaScript into your Python to get some next-level interactivity. And if you’re reading this after the fact, not to worry, it’ll be in the Rendezvous playlist on our YouTube channel.

That’s it for this week. Don’t forget to sign up for TRANSFORM!

TRANSFORM 2021 is coming

Mark your calendar: 16 to 23 April. The virtual conference for the digital subsurface is returning, a little earlier this year, so that it can incorporate the Annual General Meeting of the Software Underground on Wednesday 21 April. It will be a rather momentous occasion, because it is the first AGM since we incorporated the society last spring. Please come and help us celebrate — and determine the future of this organization!

If you just want to get to the sign-up, click the button. If you want to learn more, read on!

As last year, the conference will focus on expanding the horizons of our members. This means helping people acquire new skills, meet new people, and find out about new problems in applied subsurface science and engineering. Here’s how the schedule looks from a very high level:

An outline schedule for TRANSFORM 2021. We will publish the timing of these blocks in the near future. For now, it’s just a guide and is subject to change.

You may not have encountered some of these components before, especially not in a virtual world, so here’s a bit more detail:

• Hackathon — Teams of up to about 8 collaborate on all sorts of projects, from Python libraries to open data to reproducing papers. Check out the report part one and part two from last year.

• Tutorials — Get started on a new tool or skill! These 2-hour sessions will be 100% hands-on and can be consumed live or at your own pace. Check out the TRANSFORM 20 lessons.

• Unconference — This is the bit of the conference where we tackle the big questions. We’ll announce these components as we get closer to the event.

• Birds of a Feather (BOF) — Software Underground aims to be a platform to elevate other communities and projects. BOFs are where people with big ideas in common flock together.

• Annual General Meeting (AGM) — The Underground incorporated in April. Our first AGM will see our first elections and voting on our constitution. You’re invited, please sign up here.

• Lightning talks — Five minutes is not long, but it’s long enough to get a crowd stoked about your latest project or keenest insight! Watch the lightning bolts from Day 1 and Day 2 last year.

TRANSFORM 2020 brought over 700 digital subsurface professionals together to talk and learn about their craft. The virtual format means anyone can join us, wherever they are. And our Pay What You Like fee means that cost is never an issue. If you are planning to join us, sign up now and we’ll make sure you’re kept up to date.

---

💡 If your organization would like to sponsor TRANSFORM 2021 and grow the learning opportunities for scientists in our field, we’d love to hear from you! Find out more about supporting our programs.

Tech for online meetings

In pulling TRANSFORM 2020 together, the organizers (Filippo Broggini, Brendon Hall, Rob LEckenby, Victoria-Lou Devezes, and me) looked at a lot of technology options. I thought I’d share the stuff we used here, in case you’re wondering what you need for a full-on virtual conference. But first, I’ll try to give a high-level description of the sorts of events we hosted at the conference.

---

We had three main types of session to support:

Tutorials

We hosted 14 three-hour tutorials, which mostly involved one or two instructors and 10 to 150 learners. Behind the scenes, there was one ‘livestream host’, who was essentially like a TV director.

The instructors were in a Zoom meeting with the host, sharing screens with them. The host captured their Zoom meeting with OBS Studio, and streamed it to YouTube Live. All of the participants watched via YouTube, and the live stream is also preserved there.

Lightning talks

There were two lightning talk sessions, each two hours long. The set-up was similar to the tutorials, except that there were more presenters — 12 per session. They came into the Zoom just before their allotted time and left right after, so there were never more than 3 or 4 people in the Zoom at once.

Unsessions

These were not streamed to YouTube, so we were able to host everyone in one big Zoom meeting. (I had a Large Meeting add-on ‘just in case’ but we didn’t need it in the end; the most we had was 97 people.) This made the session susceptible to Zoom-bombing, so we were careful about where we shared the link, and we used a waiting room — with Victoria ‘on the door’ to let people in promptly.

The unsession format, which I’ll write about another time, relied heavily on Zoom’s breakout rooms. We also used GroupMap a lot — it’s a great tool for capturing ideas from a group of people.

---

Here are the tools we used:

• Slack — the heart and mind of the Software Underground already lives in our Slack workspace. To make the multiple conversations more manageable, we made channels for all of the sessions and hackathon projects, and this worked well. During the week, 700 active members exchanged about 19,000 messages, about 50% of which were in private chat.

• Zoom — the Covid-famous video conferencing tool. It worked well for us for the host–presenter meetings in the tutorials and lightning talks, and as the main room for the unsessions and hackathon presentations. We were very worried about Zoom-bombing, and were perhaps over-cautious (some legit people found it hard to enter sessions).

• YouTube — for the tutorials and lightning talks, we streamed the host–presenter Zoom to YouTube for participants to consume. This has a few big advantages: it eliminates the Zoom-bombing risk, participants can pause and rewind the live stream, and videos go straight to YouTube afterwards. And it’s free!

• OBS Studio — a fantastic open-source tool that lets you combine images, video feeds, and audio sources into a single stream, which you can send to YouTube (or Twitch or any other streaming service). This is how we streamed the Zoom sessions. It does have a learning curve though, and certainly requires an off-screen ‘director’ to manage it all — and several practice sessions to get the workflow down.

• GitHub — is indispensable for code-sharing and source control. I think all of the hackathon projects hosted their repos on GitHub. The tool is not intuitive for new programmers though, and wrangling git and GitHub is one of the most requested help topics in our hackathons and courses.

• GroupMap — a wonderful tool for collaborative brainstorming. It definitely needs a couple of hours to get the hang of what it can do, but for me this was the standout discovery of the event. Its best feature is that you can set up a workflow, like Survey > Brainstorm > Vote > Results and then guide the group through the stages, live.

• Sched and Eventbrite — for event registration. Both of these tools have their high-points — Sched is really nice for building the event schedule — but the registration process for the event was a bit of a hairball and while these tools are supposed to play nicely together, I never felt comfortable with either of them.

• Printful — a T-shirt (and other merch) vendor. The advantages are that they print ‘direct to garment’ (i.e. there’s no setup, they just print a shirt when you order one), they take care of fulfilment, and their system works seamlessly (sort of) with Squarespace, our website host. But the system is not that easy to use and I’m thinking of switching to Teespring.

• We also used a bit of hardware. Microphones were hard to find during the Covid-19 crisis, but we sent Blue Yeti Nano or Blue Snowball Ice USB mics to our instructors. The Yeti Nano is especially nice, with two pickup patterns and a hardware mute button.

• Other… The organizers and the participants used other tools during the conference, including: Mentimeter, Google Maps, Miro, and HackMD, plus of course Twitter and other social media.

---

I don’t think I appreciated it before the event — looking back, it seems obvious — but technology is completely intertwined with an online-first event. Sometimes the limitations of the tool force you to adapt; other times, the tool can enable new things you hadn’t thought of. It’s going to be interesting watching these remote collaboration tools evolve over the coming months and years. And even more interesting thinking about what kinds of meetings we can have with them!

If you have a favourite collaboration tool I haven’t mentioned here, or you’ve been in a meeting that did something different, I’d love to hear about it.