Build versus Buy - The dangers of both. The promises of a hybrid approach

Recently I was tasked with exporting the latest submissions from a specific form built with Webforms, the Drupal module. The requirements are that it runs every 30 minutes, is built in a specific XML format, and only sends submissions that have yet to be sent.

I started down the path of looking for native functionality in Webforms to do this. It has a submission export in Drush that I can use. So I figured it would use that. But I ran into a problem in that it didn’t support the XML format I needed. After some digging into the code, I found that I could create a Plugin class to define this. After a bit of work, I was able to write this.

Then I had to write the code to send to the client’s sFTP server. Because a lot of the native Drupal tools (and PHP tools in general) make working with sFTP challenging, I decided to use the PHP library that Laravel uses for this, Flysystem. More specifically the Flysystem sFTP bundle can be used separately from the full Flysystem package. This made it super easy to push files up to the client’s server.

At this point, I figured I would just write a cron job on the server that used the Webform Drush command to export the submissions to a file, using the same filename each time and replacing the contents. Then I would make a second call to my own custom Drush command to push to the sFTP server. Only I was still missing the last piece. Being able to only send unsent submissions.

Webforms don’t seem to track this, nor allow for a way to do this directly. No hooks in this process interfere. And I could not use a database query alter hook because Webforms doesn’t add query tags to this query (and probably all of its queries). After poking around a bit, I realized that I had to roll my own submission export.

At first, I tried just remaking the exporter class that Webforms had. But that was cumbersome and I couldn’t spare a ton of time. Then I came to the epiphany that I should just copy out the portions of Webform’s query generation that I need for this and put in the conditions that I needed. With that, I could just leverage the Drush command class I already created for this and it, in some ways, simplified my implementation. Now only one command instead of two. I could check anything and everything I needed to. And I didn’t need to clutter the system with submission export XML files that contain PII (personally identifiable information) which is a security risk that needs to be mitigated.

As I went through the above, I went from trying to use a pre-existing solution (buying) to creating my own (building). When I finally settled on building, I still leveraged Drupal for building the query SQL as well as storing vital config (used both for FTP settings and storing info on the latest exported) and I leveraged the simplicity of the Flysystem sFTP library to simplify my life (buying).

By taking a blended approach, I was able to accomplish more than sitting with just one or the other. And it optimized my time and the client’s needs.