When you reach a trident in the road, take it

Many thanks to Bill Dollins, Craig Williams, and Randal Hale, whose generous counsel and thoughtful insights continue to guide me along my GIS journey.

I am facing the desktop GIS trident: ArcMap, QGIS, ArcGIS Pro. Three tines. Which one should I take?

In my earlier cycling metaphor I compared selecting bicycle pedals to selecting GIS software tools. The pedals debate is over (Speedplay won); the GIS debate still rages on. I have analyzed and overanalyzed. I have consulted with the wisdom of crowds (Twitter) and the wisdom of individuals (Twitter DMs). I have considered various platforms, data formats, and multiple combinations and permutations thereof. It is time to wrap it up.

“I hear and I forget. I see and I remember. I do and I understand.” –Confucius

I begin with a 30-day challenge: use nothing but ArcGIS Pro for all my day-to-day GIS activities. I will document my experiences — good and bad — and I will update this post continuously with a running tally of said experiences. I will sum it all up on March 15.

Here we go.

  • DAY ONE [1], February 14, 2019: The challenge begins. QGIS and ArcMap and ArcCatalog icons stare at me from the Windows 10 taskbar. I do not click on them, which feels weird. I click on the ArcGIS Pro icon next to them. I am greeted with a warning: “Pro Advanced will expire in 14 days.” Stress levels go up. We always pay our bills, but sometimes we are late. What will happen on March 1 if our P.O. doesn’t make it to Redlands in time? ArcMap and QGIS never give me that kind of stress. At least all my data is on our local network. I click OK. ArcGIS Pro 2.3 launches. I single-click on a recent project. Project loads. I see my map. Some of my data is in a file geodatabase (FGDB), some in shapefiles (SHP). At some point I will need to replicate a Desktop process to periodically export a FGDB feature class to SHP and manipulate said SHP (delete and add fields) before I upload the SHP into a proprietary system. I’ll also need to replicate a separate process where I join an SHP to an Access database. I’ll attempt all of that next week. Today I’ll just tackle some light tasks. I create a custom basemap from custom aerial photography. It works. I merge two polygons. The process is smooth, similar to ArcMap’s. I hit the Save button. Does it save the project, or the merged polygons?
  • DAY TWO [2], February 15, 2019: “Pro Advanced will expire in 13 days.” Okay then. I open the same project. My custom base map is gone, even though before shutting down yesterday I clicked every Save button I could find. What did I miss? OK, it’s right there in the documentation — the custom base map has to be saved in the project you want to use it in. You can’t share custom base maps across projects. I recreate the same custom basemap in the current project. On to Catalog. Add Folder Connection works just like on Desktop. It does not auto-refresh, so if you add a new file to a connected folder you won’t see the file until you hit Refresh. Deleting fields in SHP took considerably longer than the same process in Desktop, but not a big deal for me. Adding fields is similar to Desktop, while the ability to add more than one field at a time is a welcome improvement. Now let’s join this puppy to an MS Access table. Wait, what? No MS Access support in Pro? I found a discussion on an Esri support site, basically saying don’t use Access. I beg to differ. One of my workflows is constrained by hard external factors, and one of these factors is “use Access”. Another is “use SHP”. So I will continue to use Access and SHP. But I should be able to find a workaround. Export Access table do DBF doesn’t work because long field names get truncated, effectively resulting in trying to have the same field name more than once in the same table, which DBF doesn’t like. Export to Excel and then join SHP to XLSX works. Whew! The world’s most popular database saves the day.
  • DAY THREE [6], February 19, 2019: “Pro Advanced will expire in 9 days.” Got it. I click OK. ArcGIS Pro crashes. Should I send a report to Esri? I don’t always do, but I decide to send a report this time. Report sent. Launching Pro again. “Pro Advanced will expire in 9 days.” OK. Pro runs. First I address a blog comment question from Nancy von Meyer: “Can you make a starting project with your custom base map and save that with just the base map then open it and save as something different and have the custom base map in the saved as project?” I try this. Yes, you can, and this is a convenient way to save and reuse your custom basemap. Thank you for the suggestion, Nancy! Next I decide to tackle a monthly GDB maintenance task — “Compact”. (There is also “compress”; find the difference between compact and compress on your own.) It is unclear whether it’s safe to run Compact while having the same GDB in a map in the same project. I decide to run the process on a scratch GDB and see if something blows up. Compact runs, nothing blows up. I reindex a feature class in the scratch GDB. Nothing blows up. Save project. Close Pro. Open Pro. All my stuff is still there — custom base map, recently-compacted and reindexed scratch GDB, production GDB. On to compact and reindex the production GDB. Smooth sailing. (In recent weeks I have been getting error messages while editing and saving this same GDB in ArcMap. Errors were blamed (incorrectly, IMHO) on bad network connection. Very curious to see whether the same errors on the same GDB will crop up in Pro. So far they have not. Good!)
  • DAY FOUR [7], February 20, 2019: Day of routine tasks, already written up in prior days. No surprises — good or bad. Pro just works.

On bikes and GIS

On bikes and GIS

I have been thinking a lot about bikes lately. Specifically, about transitioning from a cross bike to a road bike. More specifically, about transitioning from cross pedals to road pedals. Even more specifically, about the difference between Speedplay and Look pedals. To narrow it down even further, about why most pro cyclists ride on Look if Speedplay is supposedly better. Decisions, decisions… Can be overwhelming, and can lead to paralysis by analysis.

But in the grand scheme of things these details are nearly irrelevant. The marginal (and often dubious) performance improvement that this or that component affords the rider is almost always dwarfed by other, much more decisive factors: training, tactics, road conditions, training, training. The pros ride on whatever they ride because this is what they are being paid to ride. Their job is to RIDE THE FRIGGIN’ BIKE, so they focus on that, not on the pedals.

What does that have to do with GIS? For me, a lot.

I have been thinking a lot about GIS tools lately. Is QGIS better than ArcGIS? Is PostGIS better than file geodatabase? Is the shapefile the vinyl of geo? Should I keep my data in PostGIS and edit in QGIS, or should I use GISquirrel and edit PostGIS in ArcMap, or should I get Server and SDE and do it all in MSSQL? Decisions, decisions… Can be overwhelming, and can lead to paralysis by analysis.

But in the grand scheme of things these details are nearly irrelevant. The marginal (and often dubious) performance improvement that this or that GIS tool affords the GIS pro is almost always dwarfed by other, much more decisive factors: experience, training, environment, experience, experience. In the end it is important to remember that I am being paid TO DO THE FRIGGIN’ GIS, so I should focus on that, not on the tools.

So I will.


Ultimately, I resolve to think more about bikes and less about GIS. Speaking of which, what is the best lube for Speedplay cleats?

Shapefile, the vinyl of geo

Shapefile, the vinyl of geo

So I made this t-shirt to celebrate my love for the shapefile — the vinyl of geo. I was promptly labeled a luddite.

I am not a luddite. Far from it. I store all my data in PostgreSQL, and use PostGIS and QGIS like any self-respecting geohipster. I dabble in Python and JavaScript. I write SQL queries.

But I also live in the real world. I make maps and print them. I export them to PDF. I exchange data with other users and organizations — most of them casual GIS users. And these users (and their applications) only know and understand the shapefile.

Casual GIS users will continue to use the shapefile because it is simple, convenient, and universally supported. It just works. To call for the abolition of the shapefile is akin to calling for the abolition of the .xls(x) format on the grounds that people erroneously use it in lieu of “legitimate” databases. Millions of people.

Let’s be real and call off the shapefile vendetta. There is room (and need) in the geoverse for both PostGIS and shapefile. Why not just buy yourself a “Postgres is my databae” or “I ❤ .SHP” t-shirt to show where you stand? You’ll look hip, whichever you choose.