Pither.com / Simonhttps://www.pither.com/simon/blog/2015-09-25T16:18:19+00:00Development, systems administration, parenting and businessGrails 3 Externalized Configuration2015-09-25T16:11:44+00:002015-09-25T16:18:19+00:00Simon Pitherhttps://www.pither.com/simon/blog/2015/09/25/grails-externalized-configuration<article>
<h1>Grails 3 Externalized Configuration</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2015-09-25T16:11:44+00:00">Fri 25th Sep 2015</time>
</div>
<div>
<p>Grails 2.x uses it's own configuration loader that includes the ability to read extra configuration from additional, external files. This is a great feature for reading production, deployment specific configuration values; while keeping them separate from a standard WAR file. I've found this very handy when the same application needs to be deployed to multiple machines with only some configuration differences.</p>
<p>To do this in Grails 2.x you can use a line similar to this in your Config.groovy file:</p>
<pre><code>grails.config.locations = [ 'file:/etc/my_path/config_file.groovy' ]
</code></pre>
<p>I recently tried to do the same thing with a Grails 3.x application and discovered that the configuration loading is now powered by Spring Boot - the grails.config.locations option is no longer used at all.</p>
<p>However this is generally great for simplifying Grails and the Spring Boot config loader does actually include an awful lot of <a href="http://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html">options, including some for loading external (from the WAR) configuration</a> files. Unfortunately all of the locations it uses are very much within the Java deployment area and weren't really what I wanted.</p>
<p>With a little help from a kind soul on the <a href="https://grails.org/community.html">Grails Slack channels</a> I discovered that this problem had already been <a href="http://grails.1312388.n4.nabble.com/Grails-3-External-config-td4658823.html">discussed on the Grails mailing list</a> and a couple of possible code snippets were included for loading an external config file.</p>
<p>So given those examples and my specific requirement to load some configuration values from an external file, where the location of that file is pre-set within the application, I ended up making my Application.groovy class look like the code below. This code has a hard coded base file path (and supports overriding it) set to "/etc/my_path/config-file" and will look for both a .yml and .groovy version of that path (ie "/etc/my_path/config-file.yml" and "/etc/my_path/config-file.groovy"); if found they'll be loaded as configuration sources.</p>
<pre><code>import grails.boot.GrailsApp
import grails.boot.config.GrailsAutoConfiguration
import org.springframework.beans.factory.config.YamlPropertiesFactoryBean
import org.springframework.context.EnvironmentAware
import org.springframework.core.env.Environment
import org.springframework.core.env.MapPropertySource
import org.springframework.core.env.PropertiesPropertySource
import org.springframework.core.io.FileSystemResource
import org.springframework.core.io.Resource
class Application extends GrailsAutoConfiguration implements EnvironmentAware {
static void main(String[] args) {
GrailsApp.run(Application, args)
}
@Override
void setEnvironment(Environment environment) {
def configBase = System.getenv('GRAILS_CONFIG_LOCATION') ?: System.getProperty('grails.config.location') ?: "/etc/my_path/config-file"
boolean configLoaded = false
def ymlConfig = new File(configBase + '.yml')
if(ymlConfig.exists()) {
println "Loading external configuration from YAML: ${ymlConfig.absolutePath}"
Resource resourceConfig = new FileSystemResource(ymlConfig)
YamlPropertiesFactoryBean ypfb = new YamlPropertiesFactoryBean()
ypfb.setResources(resourceConfig)
ypfb.afterPropertiesSet()
Properties properties = ypfb.getObject()
environment.propertySources.addFirst(new PropertiesPropertySource("externalYamlConfig", properties))
println "External YAML configuration loaded."
configLoaded = true
}
def groovyConfig = new File(configBase + '.groovy')
if(groovyConfig.exists()) {
println "Loading external configuration from Groovy: ${groovyConfig.absolutePath}"
def config = new ConfigSlurper().parse(groovyConfig.toURL())
environment.propertySources.addFirst(new MapPropertySource("externalGroovyConfig", config))
println "External Groovy configuration loaded."
configLoaded = true
}
if(!configLoaded) {
println "External config could not be found, checked ${ymlConfig.absolutePath} and ${groovyConfig.absolutePath}"
}
}
}
</code></pre>
<p>This code attempts to make the external configuration values the highest priority, so that they override any values pre-set within the application. This seems to work generally but unfortunately doesn't for any internally specified values within environment blocks.</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=groovy">groovy</a>
</div>
</div>
</article>
First Grails 3 pre-release2015-01-30T22:07:58+00:002015-01-30T22:08:09+00:00Simon Pitherhttps://www.pither.com/simon/blog/2015/01/30/first-grails-3-pre-release<article>
<h1>First Grails 3 pre-release</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2015-01-30T22:07:58+00:00">Fri 30th Jan 2015</time>
</div>
<div>
<p>The first <a href="https://github.com/grails/grails-core/releases/tag/v3.0.0.M1">Grails 3 pre-release</a> went live a few days ago.</p>
<p>Soon after I updated my <a href="http://www.pither.com/simon/blog/2011/03/13/grails-multi-version-launch-script">super simple grails download and launch script</a> (<a href="https://github.com/spither/grails-multi-launch">on github</a>) with the new download locations and other small tweaks for Grails 3 support. Grab the updated script from github then try out Grails 3 with:</p>
<pre><code>grails 3.0.0.M1 create-app my-test-app
</code></pre>
<p>Grails 3 comes with a lot of big changes, including lots of improvements to testing, test run speeds and general startup speed. Check out the <a href="https://www.youtube.com/watch?v=aro3_RZqgtU">preview video</a> for more new features. I'm really hoping that these improvements might be able to improve the <a href="http://www.pither.com/simon/blog/2015/01/30/grails-vs-django-revisited">score I recently gave Grails</a>.</p>
<p>There is also a write up about using the excellent <a href="http://davydotcom.com/blog/2015-01-29-grails-3-0-0-m1-asset-pipeline-tips-tricks">asset-pipeline plugins with Grails 3</a>.</p>
<p>Unfortunately my first attempt at running Grails 3 managed to hit <a href="https://issues.gradle.org/browse/GRADLE-3076">a Gradle bug</a> that stops it working if your /tmp is mounted with noexec. It seems the suggested work-arounds to get Gradle to use a different temp directory no longer work, so the only solution I found was to temporarily remount /tmp with exec permissions:</p>
<pre><code>sudo mount /tmp -o remount,exec
</code></pre>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
Grails vs Django, revisited2015-01-30T12:00:00+00:002015-01-30T16:58:22+00:00Simon Pitherhttps://www.pither.com/simon/blog/2015/01/30/grails-vs-django-revisited<article>
<h1>Grails vs Django, revisited</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2015-01-30T12:00:00+00:00">Fri 30th Jan 2015</time>
</div>
<div>
<p>Some while ago now, I considered the question of <a href="http://www.pither.com/simon/blog/2013/08/02/grails-vs-django">Grails vs Django</a> as my web development framework of choice. Since then I've <a href="http://www.patma.co.uk">created</a> <a href="https://battleships.allsecuredomains.com/">a few</a> Django web applications. I've also continued to work on several Grails based projects, <a href="http://www.breakoutbrasil.com/breakoutBrasil2014/vote">new</a> <a href="http://www.supajam.com/">and</a> <a href="http://www.infiniti-gp.com/">old</a>.</p>
<p>So with fresh battle experience of both frameworks I thought I'd revisit my previous thoughts to see what I feel needs changing, or not, from my early comparison. You can skip to <a href="#tldr">the conclusion</a> if you want.</p>
<h2>Development Ease</h2>
<p>Previous: close but a win for Django.</p>
<p>Now: this is definitely a win for Django.</p>
<p>Being able to run a full test suite from scratch in less time than it takes Grails just to start the JVM is such an incredible boost to concentration and hence productivity. Both systems auto-reload changes and both seem to stumble on this fairly frequently (especially with in place plugins for Grails), but again Django's immensely faster full-restart time saves it.</p>
<p>Both work equally well with IntelliJ IDEA and vim, so I can work with either in both of my main development environments.</p>
<p>Many, many other things are different between the two but not such that they give either an advantage over the other.</p>
<h2>Technical Features</h2>
<p>Previous: Grails won for it's family of resource plugins.</p>
<p>Now: it's much closer, I think just about still in favour of Grails.</p>
<p>Last time I talked about Django admin pages and the Grails resource plugins. With experience, I've hardly used the Django admin pages, although they are nice when I have (and Alexandre pointed out Grails has <a href="http://grailsadmininterface.kaleidos.net/">a plugin</a> doing similar); and Grails has replaced the resource plugins with asset-pipeline plugins.</p>
<p>In the more distant past I've lost many an hour to fighting the old Grails resource plugins, including submitting patches, only a few of which got included. In the last year I've spent many an hour fighting the new Grails asset-pipline plugins, including submitting patches, all of which got included. On a slight Grails aside, I found some pretty major issues with the new asset-pipeline plugins but at least debugging them was much simpler and the maintainer has been a lot more responsive too.</p>
<p>However that's all rather a distraction really because the only point is that Grails has (mostly) working, out of the box static resource handling (caching, compressing, etc); where Django does not. In fact, while I'm aware of a couple of Django plugins that should provide the functionality, I've actually not got around to investing the time to get them working. While my resulting web applications still work, they are definitely poorer without this. I will be addressing this soon, but currently it's a strong negative for Django.</p>
<p>It isn't all bad news for Django though. While working with it I've found it does include an awful lot of little niceties, some cherries on the cake. There are several built in template filters that I've had to write several times myself for Grails. I've had a number of occasions, not just in the templates, where I've thought, "oh, excellent, I don't have to write that myself"; which is always pleasant to discover.</p>
<h2>Testing</h2>
<p>Previous: a draw.</p>
<p>Now: an easy Django win.</p>
<p>The latest versions of Django now include a better test running setup out of the box which removes pretty much my only complaint. Writing tests in Django works well, feels solid, predictable and includes lots of nice helpers.</p>
<p>For Grails, testing continues to feel like hard work. I've continued to spend time in the last year debugging tests only to find the problem is a quirk of the Grails testing system. Yes this is getting better with newer versions and it's a specific area the Grails team are working on but currently they just aren't there yet.</p>
<p>The only thing I really do miss from Grails testing is Spock but even the great Spock syntax and tools aren't enough to rescue this one for Grails.</p>
<h2>Libraries</h2>
<p>Previous: a win for Grails (thanks to huge numbers of Java libraries).</p>
<p>Now: a draw.</p>
<p>It may still be the case that there are more things covered by Java libraries than Python, but everything I've needed so far has been readily available; so with experience I'm going to call this a draw.</p>
<h2>Developers</h2>
<p>Previous: a draw.</p>
<p>Now: still a draw.</p>
<p>I don't think the situation here has changed much.</p>
<h2>Deployment</h2>
<p>Previous: a win for Django.</p>
<p>Now: very close but revised to a draw.</p>
<p>All of the points I made before still stand - Django is lighter weight and quicker; Grails is simpler to deploy (a single WAR file) and benefits from the JVM.</p>
<p>The changed result is because I've now put several Django apps into production and I really missed the simplicity of WAR deployment. Being able to package everything up in advance into a nice neat bundle and then deploy the identical binary to identical servers and get identical results is really good. I'm yet to discover this deployment simplicity with Django. In fact, Django deployments so far have seemed a bit unpleasant - creating and maintaining Python virtual environments and running Django scripts (in the correct environment) on the machine to collect assets or migrate databases has just felt a bit cumbersome.</p>
<p>I still really like the smaller, quicker Django for smaller deployments and quite dislike the bulky, slow Grails even for big machine, cluster deployments; but the single bundle file deployment has just about pulled it back to a draw.</p>
<h2>Experience</h2>
<p>Previous: a win for Grails.</p>
<p>Now: a draw.</p>
<p>With a year of gaining Django experience on smaller projects, I've been very pleased with how smoothly it's gone. There have been a couple of odd errors that took a while to track down, but nothing I've had to use a debugger to find (which I do seem to need semi-regularly with Grails).</p>
<p>I still feel more comfortable finding my way around a Grails project (understandable as I have a lot of years of history with it) and that is why Grails is just about holding this one to a draw, even though I'm pretty quick and happy working in either framework now.</p>
<h2><a id="tldr"></a> Final Conclusion</h2>
<p>The updated results are:</p>
<ul>
<li>Development Ease: Django</li>
<li>Technical Features: Grails</li>
<li>Testing: Django</li>
<li>Libraries: draw</li>
<li>Developers: draw</li>
<li>Deployment: draw</li>
<li>Experience: draw</li>
</ul>
<p>Which makes the final scores (excluding the draws): Django 2 and Grails 1.</p>
<p>The general trend in these results is that I feel the contest is very much closer than I did before. However the actual numbers have now flipped the winner, just about brining Django out on top!</p>
<p>As I did before, I still think that Django and Grails are very closely matched and if you need to make a choice, basing it on existing infrastructure, language experience or even just flipping a coin will always leave you with a very good and enjoyable web framework to work with.</p>
<p>For me personally, I'm especially enjoying the ease of development from working with Django and I'm going to continue working with it, including making use of it for some much larger client projects in the near future.</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
<a rel="tag" href="/simon/blog/?tag=django">django</a>
</div>
</div>
</article>
Design a Formula 1 Helmet2013-09-15T00:00:00+00:002013-10-10T20:35:09+00:00Simon Pitherhttps://www.pither.com/simon/blog/2013/09/15/design-a-formula1-helmet<article>
<h1>Design a Formula 1 Helmet</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2013-09-15T00:00:00+00:00">Sun 15th Sep 2013</time>
(updated <time datetime="2013-10-10T20:35:09+00:00">Thu 10th Oct 2013</time>)
</div>
<div>
<p>One of the projects that has been filling, a little too much at times, my recent non-parenting hours is now accepting entries.</p>
<p>The site is an addition to the <a href="http://www.infiniti-gp.com/">Infiniti Formula 1 press site</a> to provide competitions for people to design Formula 1 helmets (or a small non-advertiser section of them) that Sebastian Vettel will wear later in the season.</p>
<p>If you think you'd make a good Formula 1 helmet designer then Infiniti are running a specific <a href="http://www.infiniti-gp.com/competition/vettel-austin-helmet">US competition</a> plus a different one <a href="http://www.infiniti-gp.com/competition/vettel-abu-dhabi-helmet">for everyone else</a>.</p>
<p><img alt="Infiniti GP competition homepage" src="/files/infiniti-helmet-home.jpg" /></p>
<p>The site was written using Grails and is generally pretty standard stuff with just a little bit of image processing on uploads and within the competition admin system. However there is one much more interesting area of the site, namely an online (Javascript powered) image editor.</p>
<p><img alt="Infiniti GP competition javascript image editor" src="/files/infiniti-helmet-editor.jpg" /></p>
<p>The javascript image editor is a customised version of <a href="https://github.com/PlayMyCode/SkyBrush">the excellent SkyBrush editor</a>. Although it took some changes to get it working the way we needed the basic integration went well and I think the editing features provided are excellent.</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=javascript">javascript</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
Grails vs Django2013-08-02T00:00:00+00:002015-01-30T20:07:46+00:00Simon Pitherhttps://www.pither.com/simon/blog/2013/08/02/grails-vs-django<article>
<h1>Grails vs Django</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2013-08-02T00:00:00+00:00">Fri 2nd Aug 2013</time>
(updated <time datetime="2015-01-30T20:07:46+00:00">Fri 30th Jan 2015</time>)
</div>
<div>
<p><em>Update:</em> I revisited this <a href="http://www.pither.com/simon/blog/2015/01/30/grails-vs-django-revisited">Grails vs Django</a> discussion in Jan 2015.</p>
<p>Before I met Grails I had spent several years writing my web applications in Perl, including work on two different in-house frameworks to try to make life easier! I also dabbled a little with Ruby on Rails but for some reason it just didn't stick (I think I prefer brackets to words!). Then I discovered <a href="http://grails.org/">Grails</a> (on version 0.3 at the time I think) and the increase in brackets and reduction in semi-colons, thanks to Groovy, fitted me well.</p>
<p>So in the years since then I <a href="https://www.allsecuredomains.com/">have</a> <a href="http://www.breakoutbrasil.com/breakoutBrasil/vote">written</a> <a href="http://www.supajam.com/">several</a> <a href="http://www.infiniti-gp.com/">sites</a> using the Grails framework.</p>
<p>Then in a recent discussion with my <a href="http://www.seoss.co.uk/">SEOSS</a> business partner, Tim put forward the idea of using <a href="https://www.djangoproject.com/">Django</a> on a project we were quoting for. It's been a while since I last wrote any Python code and I've never used Django before. So I ended up doing some research and put together the following report.</p>
<p>There is a conclusion near the bottom if you feel it's just too long. <strong><a href="#tldr">TL; DR - scroll to the end!</a></strong></p>
<h2>Development Ease</h2>
<p>Feature wise I think they are very evenly matched for development. Compared to Java, Grails adds a fantastic automatic reloading of changed classes and components. However I'm not comparing it to Java at the moment and from years of use I also know that the Grails re-loading has it's rough edges and when you do need a full restart, it's slow, very slow. Similarly test runs can suffer from the same extra startup time baggage, eating precious time and concentration.</p>
<p>In contrast Django has almost instant startup time and also includes auto reloading of changes.</p>
<p>Text editor/IDE support, debugging, logging and anything else I can think of they seem pretty evenly matched on, so this one is close but is a win for Django.</p>
<h2>Technical Features</h2>
<p>The two frameworks take quite different approaches to many things, and name almost everything slightly differently (just to make comparison that little bit easier!) but in one way or another they both seem to cover most of the standard stuff that I'd expect in such a framework. My investigations so far have picked up a couple of differences I want to highlight though.</p>
<h3>Admin Pages</h3>
<p>Django provides a very nice looking facility to quickly and easily auto-build admin pages that are pre-styled and of production quality. I suspect the limitation is when you find a little extra logic is required in such a page and that isn't allowed for in the auto-built pages, but for many cases this looks really good. The big question is how fast a project would outgrow these pages, or perhaps whether they are actually suitable at all. Most sites will need at least a limited admin system of this type but for most of our projects the vast majority of the development isn't in the admin system anyway.</p>
<h3>Static Resources</h3>
<p>Both frameworks do cover collection, grouping and publishing of these files (CSS, Javascript, images) but Grails includes versioning (a hash of content) of the file names so that they can be served as cache-for-ever. Django does not appear to include this, although after much searching I think I've found a plugin that does.</p>
<p>Pre-processing to cover things like LESS or optimisation of CSS and JS also seem to only be in hard to find and slightly unclear plugins for Django. The Grails implementations of these are from the excellent resources family of plugins, the base of which now comes installed by default in new projects.</p>
<h3>Conclusion (Technical Features)</h3>
<p>While I really like the idea of the admin pages, I think they would probably be outgrown quite fast in most cases and I can't imagine living without the resources plugins so I think it's a win for Grails.</p>
<h2>Testing</h2>
<p>Heavy testing is strongly encouraged by both frameworks. Grails comes with JUnit and also supports Spock relatively smoothly via a plugin. The default testing system for Django appears to be universally recommended for immediate replacement with another Python testing library, but the process seems easy.</p>
<p>Overall both platforms seem to have good support for unit, integration and functional tests. I know from experience there are some edge cases around GORM support in unit tests that can be frustrating but without experience I don't know how Django fairs for this.</p>
<p>I really like Spock tests so that's a definitely positive for Grails but on the surface Django seems to support everything needed for good, quick test development so I think it would be unfair to mark it down. I'm going to call this one a draw.</p>
<h2>Libraries</h2>
<p>Django has more plugins listed (although I still couldn't find any covering some things I searched for) but Grails can fall back on the enormous base of Java libraries which can be easily included from Maven repositories or manually as jar files.</p>
<p>In practice Django and Python probably have everything needed for most projects but just for the huge number of Java libraries and their very easy inclusion, this has to be a win for Grails.</p>
<h2>Developers</h2>
<p>This category is to consider the availability of future developers for work we create and for us hiring in assistance on larger projects.</p>
<p>The Django community is definitely larger than Grails, but the Java community is in turn much, much larger than Python. Grails itself is far from small these days with numerous very large companies using it and with many developers available. In my role at SupaJam I have hired permanent Grails developers a few times and always had plenty of applicants. The Brighton Java group even ran a <a href="http://www.meetup.com/Brighton-Java/events/126157102/">recent event on Grails</a> (unfortunately I couldn't go as it clashed with parents evening).</p>
<p>I don't think there is much difference in the permanent job market but for freelance or contract it may be the case that Grails developers cost a bit more than Django ones, although not by a huge amount.</p>
<p>I'm going to call this one another draw.</p>
<h2>Deployment</h2>
<p>I think this is the most complex category as production deployments vary a lot anyway. Taking just a simple deployment the two platforms are still very different in many ways. I both love and hate Grails (Java) deployments so I've got some specific points that I want to compare them on. I have tried to list the positives for each (obviously each one is inherently a negative for the other framework).</p>
<h3>For Django</h3>
<ul>
<li>Much smaller memory footprint</li>
<li>Less moving parts. Typically just Apache or Nginx with a module, rather than a separate service. Although larger deployments would probably run a separate daemon and of course load balancing would effectively force a separate service anyway.</li>
<li>Much faster restart to pick up changes (ie Apache restart vs Tomcat restart)</li>
<li>Due to faster restart, less release down time and/or easier release process scripting (especially in clusters with load balancers)</li>
</ul>
<h3>For Grails</h3>
<ul>
<li>Very easy deployment packaging (ie a WAR file)</li>
<li>Better support for starting new threads and background tasks</li>
<li>Built-in (ie managed within application code base) background/cron tasks</li>
<li>In-JVM state and caching - faster and simpler, eg avoids the need to run memcached</li>
</ul>
<h3>Conclusion (Deployment)</h3>
<p>I think this is my toughest call in all of the categories as I really like the simplicity of WAR files, in-JVM state and in-code-base crons. However the large and slow deployment process is probably the thing I like least about Grails. Overall I'm going to call this one in favour of Django.</p>
<h2>Experience</h2>
<p>This is obviously a win for Grails. The question here is more about how much this is worth. It is very easy to find reports online for both Grails and Django bugs causing people pain and certainly from personal experience I know framework bugs can be expensive when hit for the first time. Beyond that there will always be learning time required for any new framework, especially one with some radically different approaches.</p>
<h2><a id="tldr"></a> Final Conclusion</h2>
<p>The results are:</p>
<ul>
<li>Development Ease: Django</li>
<li>Technical Features: Grails</li>
<li>Testing: draw</li>
<li>Libraries: Grails</li>
<li>Developers: draw</li>
<li>Deployment: Django</li>
<li>Experience: Grails</li>
</ul>
<p>Which makes the final scores (excluding the draws): Django 2 and Grails 3.</p>
<p>Django won for development ease and deployment which are both very important categories, although they were only slim wins. Two of the Grails wins were also quite slim and the third was experience which will dissipate over time. So overall I think they are very closely matched.</p>
<p>For SEOSS the decision was clear. There is nothing in Django that would be expected to counter the huge benefit of our existing experience, at least in the short term. The only option we could sensibly quote on was Grails.</p>
<p>It is worth noting that all of the Django information presented here is from reading things online (the documentation, tutorials, plugin searches and more documentation) and creating a one-line hello world app. So it's probably missing important details, both for and against the use of Django, that can only be found by using it in anger. My aim was to make a decision for a quote we were preparing, not to settle any wars or make life long choices.</p>
<p>Where would the fun be in technology if there weren't always new things to learn. This report may have provided a win for Grails, but it has also piqued my interest in Django. As it turns out I actually have a personal project that I'm just scheduling some time for, so I think it may just become my Django testing ground! With a little luck I'll write up how I get on.</p>
<p>Update: <a href="/articles/2014/01/07/my-first-django-site">My first Django site is live</a></p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
<a rel="tag" href="/simon/blog/?tag=django">django</a>
</div>
</div>
</article>
Add an SMTP server to a Grails application2011-09-20T00:00:00+00:002013-07-29T13:50:23+00:00Simon Pitherhttps://www.pither.com/simon/blog/2011/09/20/add-an-smtp-server-to-a-grails-application<article>
<h1>Add an SMTP server to a Grails application</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2011-09-20T00:00:00+00:00">Tue 20th Sep 2011</time>
(updated <time datetime="2013-07-29T13:50:23+00:00">Mon 29th Jul 2013</time>)
</div>
<div>
<p>Quite a long time ago I talked about <a href="http://www.pither.com/articles/2010/06/22/envelope-senders-for-grails-mail-plugin-spring-java-mail">setting envelope sender addresses</a> so that I could send mailshots and get back any bounces in a useful way. The missing component to it was to process those bounces within my application. Here, finally, is my write up of how I did this.</p>
<p>After much searching I eventually found the excellent <a href="http://code.google.com/p/subethasmtp/">SubEtha SMTP</a> project which made including an SMTP server within my Grails application incredibly easy. Just four simple steps are required...</p>
<p>Include the library (grails-app/conf/BuildConfig.groovy):</p>
<pre><code>dependencies {
compile('org.subethamail:subethasmtp:3.1.4') {
excludes 'mail'
}
}
</code></pre>
<p>Write a SubEtha message listener (for me: src/groovy/com/supajam/smtp/MessageListener.groovy):</p>
<pre><code>package com.supajam.smtp;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.subethamail.smtp.TooMuchDataException;
import org.subethamail.smtp.helper.SimpleMessageListener
import com.supajam.BounceRecord
public class MessageListener implements SimpleMessageListener {
private final static Logger log = LoggerFactory.getLogger(MessageListener.class);
public boolean accept(String from, String recipient) {
return true;
}
public void deliver(String from, String recipient, InputStream data) throws TooMuchDataException, IOException {
// I use this to process bounce messages, but it's just an
// email message that could be processed however you like
BounceRecord.withTransaction {
if (log.isDebugEnabled())
log.debug("Processing mail from " + from + " to " + recipient);
// The regex on the next line is specific to my VERP scheme
def origAddr = recipient =~ /-(.+=.+)@/
if(origAddr && origAddr[0].size() > 1) {
// here I extract the source address, ensure it's
// valid, record the bounce and remove the address
// from the mailing list.
}
}
}
}
</code></pre>
<p>I've included an outline of the code that I actually use to process incoming email - all of which I know will be bounce messages. There's nothing to force this use though, it's just a generic way to receive emails into an application.</p>
<p>Wire it together with Spring (grails-app/conf/spring/resources.groovy):</p>
<pre><code>beans = {
// Create an in-process SMTP server for processing bounces
smtpMessageListener(com.supajam.smtp.MessageListener)
smtpMessageListenerAdapter(org.subethamail.smtp.helper.SimpleMessageListenerAdapter, ref('smtpMessageListener'))
smtpServer(org.subethamail.smtp.server.SMTPServer, ref('smtpMessageListenerAdapter')) {
port = 2500
hostName = "my.server.name"
disableTLS = true
maxConnections = 10
}
}
</code></pre>
<p>To avoid running my grails application as root, I set the SMTP port to a high number (2500 in this example). Getting emails delivered to such a port is beyond this post (but personally I use iptables redirects).</p>
<p>Finally, make it run at application start (grails-app/conf/BootStrap.groovy):</p>
<pre><code>class BootStrap {
def smtpServer
def init = { servletContext ->
log.debug("Starting SMTP server")
try {
smtpServer.start()
}
catch(Exception e) {
// Starting the web application is more important
// than SMTP, so just log the failure and carry on
log.error("SMTP server could not be started")
}
}
def destroy = {
log.debug("Stopping SMTP server")
smtpServer.stop()
}
}
</code></pre>
<p>I guard the SMTP server start just in case there's a problem. No SMTP server is a nuisance but no web application is very bad. Just for cleanliness I also attempt to stop it in the destroy method.</p>
<p>That's it! It really is that simple to have a full SMTP server running within grails and ready to accept and process incoming email.</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=email">email</a>
<a rel="tag" href="/simon/blog/?tag=smtp">smtp</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
Ticket sales (PayPal, Jasper Reports, Barcodes) with Grails2011-07-25T00:00:00+00:002013-07-29T13:53:16+00:00Simon Pitherhttps://www.pither.com/simon/blog/2011/07/25/ticket-sales-paypal-jasper-reports-barcodes-with-grails<article>
<h1>Ticket sales (PayPal, Jasper Reports, Barcodes) with Grails</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2011-07-25T00:00:00+00:00">Mon 25th Jul 2011</time>
(updated <time datetime="2013-07-29T13:53:16+00:00">Mon 29th Jul 2013</time>)
</div>
<div>
<p>Please see <a href="/articles/2011/07/19/tickets-selling-sending-supporting-and-checking-all-in-three-weeks">Part 1</a> for the start of this series and links to all parts.</p>
<p>For the <a href="http://www.supajam.com/">SupaJam</a> ticket outlet, we didn't need anything too fancy, just enough to present people with a couple of ticket types to choose from and a way to pay for them. Of course we still had the extra requirement of collecting a real name for each and every ticket as well though.</p>
<p>We kept the structure online pretty simple, sticking to a single web page, either re-displaying with errors or sending the user straight to PayPal (our chosen payment processor purely for reasons of technical experience).</p>
<p>So far this was all fairly basic coding, a command object, controller, a couple of views and a few new domain objects to keep it all. This is where <a href="http://grails.org/">Grails</a> really shines, making all of this simple and quick.</p>
<p>There is already a PayPal plugin for grails, but in this case I didn't feel it fitted, so the PayPal integration was custom coded. This included building and passing a cart, based on user selections as well as an IPN endpoint to make sure all the purchases were dealt with automatically.</p>
<p>Once a user has visited one of our ticket pages and decided to take up an offer, been to PayPal and paid, it's generally useful to then provide a ticket. This was the part that presented the first interesting challenge.</p>
<p>We hadn't finalised how we were going to run our festival entrance yet but it seemed like a fast method to check unique ticket codes might be handy. The quick and simple answer to this seemed like a barcode. So each person would need a physical ticket to bring along that we could scan and quickly check validity. We didn't want to add postage costs to what we charged, so email was the simplest solution. As for the most controllable and widely readable document format to send people - PDF of course!</p>
<p>So we needed to make PDFs that would contain some useful instructions to the owner, a few critical ticket details plus the essential unique code and its barcode representation. To make them look nicer and also to better fill an A4 page we also decided to add an image about the festival.</p>
<p>I had read about Jasper Reports before and knew there was a Grails plugin for it, so I decided it would be a good place to start. I soon had iReport downloaded, running and a simple ticket report designed.</p>
<p>While the basics of the ticket looked simple enough, there were two elements that proved slightly tougher. The most critical was the barcode. When adding a barcode in iReport you first get a choice of barcode library. So I researched each and picked the one I preferred.</p>
<p>This worked fine in iReport, but it soon became apparent that the Grails Jasper Reports plugin deliberately excludes most of the optional components and dependencies of Jasper. This included both of the barcode libraries. So off I went trying to add <a href="http://barcode4j.sourceforge.net/">Barcode4J</a> to my grails project. </p>
<p>After an hour or so of fighting to include one layer of dependencies while desperately excluding the next layer (mostly to avoid XML parser conflicts), I admitted defeat and had a crack at the <a href="http://barbecue.sourceforge.net/">Barbecue</a> barcode library instead. Thankfully this turned out to be trivial in comparison and just needed:</p>
<pre><code>compile('net.sourceforge.barbecue:barbecue:1.5-beta1') {
excludes 'portlet-api', 'servlet-api', 'junit'
}
</code></pre>
<p>The only downside was that I couldn't get the code to be automatically included under the barcode (even in iReport). Just including the code in a text field was a simple work around though.</p>
<p>Next up was the included images - for branding, page filling and information. I wanted to change these for different events and fetching each image from an event configured URL seemed like a good way to go. At least I thought that until I tried it!</p>
<p>For quite some time Jasper Reports stubbornly refused to load the images into the report. Test images from another site would work, but not those from my development grails server.</p>
<p>After much debugging and eventually resorting to tcpdump(!), I tracked this down to the zipped-resources plugin which currently does two bad things (as documented). It doesn't check to see if a client actually supports gzip and it gzip's images! So this meant that Jasper was being given gzipped images which it couldn't deal with.</p>
<p>Once I realised this, simply removing the zipped-resources plugin solved the problem. Although in the end I actually re-instated the zipped-resources plugin and decided to host the images on one of our external (ie not Grails) web servers.</p>
<p>My final little niggle with Jasper was font based. I could not seem to find a font that both Jasper could measure (for alignment purposes) and clients could find when rendering the PDF. I ended up solving this by using the Jasper-fonts package and choosing to embed the fonts within the PDF. This needed another line in my grails-app/conf/BuildConfig.groovy file:</p>
<pre><code>compile('net.sf.jasperreports:jasperreports-fonts:4.0.0')
</code></pre>
<p>And that was it.</p>
<p>In less than a (short, due to bank holidays) week I built an e-commerce ticket agent system that created pretty, barcoded e-tickets. This promptly went live on <a href="http://www.supajam.com/">SupaJam</a> and sold several hundred (up to thousands by now) festival tickets.</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=supajam">supajam</a>
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=jasper">jasper</a>
<a rel="tag" href="/simon/blog/?tag=reports">reports</a>
<a rel="tag" href="/simon/blog/?tag=barcode">barcode</a>
<a rel="tag" href="/simon/blog/?tag=resources">resources</a>
<a rel="tag" href="/simon/blog/?tag=zipped-resources">zipped-resources</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
Tickets: selling, sending, supporting and checking; all in three weeks2011-07-19T00:00:00+00:002013-07-29T13:52:49+00:00Simon Pitherhttps://www.pither.com/simon/blog/2011/07/19/tickets-selling-sending-supporting-and-checking-all-in-three-weeks<article>
<h1>Tickets: selling, sending, supporting and checking; all in three weeks</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2011-07-19T00:00:00+00:00">Tue 19th Jul 2011</time>
(updated <time datetime="2013-07-29T13:52:49+00:00">Mon 29th Jul 2013</time>)
</div>
<div>
<p>Towards the end of May, <a href="http://www.supajam.com/">SupaJam</a> launched the first round of several festival ticket give aways planned for this summer. This wasn’t the first time SupaJam has given away festival tickets, but it is the first time that we've done it in this way.</p>
<p>Technically, it all went well. The days afterwards did include a rather large number of support emails though!</p>
<p>In hind sight, this isn't entirely surprising, we do make life a little difficult for ourselves. These give aways are providing a limited number of quite valuable tickets, so to protect ourselves and the festivals we want to make sure that the tickets we give away are not resold. We don't want a ticket tout to end up benefiting. Hence we always collect a real (ID-able) name for each ticket. If that wasn't enough to ask of people claiming these tickets, we gave them away first-come-first-served, so everyone typing in their details was typing in a hurry!</p>
<p>So in flooded the requests to change names, correct spellings, return duplicates that were got for or by friends and assorted other amendments.</p>
<p>It quickly became apparent that giving away tickets in this way had a fairly high support cost associated with it.</p>
<p>So we put our thinking caps on and decided that we should charge a small administration fee to cover some of the support costs that seemed to be inevitable. Of course, this meant we needed some kind of e-commerce setup in order to process these fees.</p>
<p>Part of the deal that we made with the festival for these tickets meant that we had to run our own box office, on site at the festival, to process and verify each of our ticket holders. A process for making that happen, both reliably and quickly was also a challenge we hadn't solved yet.</p>
<p>By the time we'd revised our approach, we had just three weeks until the first festival - the <a href="http://londonfeis.com/">London Feis</a> in Finsbury Park. This story (which I've split into four further parts to ease my writing as much as your reading) is my journey through the technology solutions I developed and built to enable hundreds of people to enjoy a free festival courtesy of SupaJam.</p>
<p>(Each part below will become a link as I write the articles.)</p>
<p><a href="/articles/2011/07/25/ticket-sales-paypal-jasper-reports-barcodes-with-grails">Part 2 - Ticket sales</a><br />
Part 3 - Customer support and search<br />
Part 4 - Creating and running the technology for a box office<br />
Part 5 - Lessons from becoming a ticket agent and box office in 3 weeks</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=supajam">supajam</a>
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
grails-mail patch for envelopeFrom (VERP)2011-03-13T00:00:00+00:002013-11-19T14:08:44+00:00Simon Pitherhttps://www.pither.com/simon/blog/2011/03/13/grails-mail-patch-for-envelopefrom-verp<article>
<h1>grails-mail patch for envelopeFrom (VERP)</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2011-03-13T00:00:00+00:00">Sun 13th Mar 2011</time>
(updated <time datetime="2013-11-19T14:08:44+00:00">Tue 19th Nov 2013</time>)
</div>
<div>
<p>I have previously written about <a href="/articles/2010/06/22/envelope-senders-for-grails-mail-plugin-spring-java-mail">setting envelope senders when sending email in grails</a> and following comments made on that post I did actually get around to submitting a patch to the plugin.</p>
<p>However a change to that patch was suggested and before I managed to implement it, the mail plugin underwent a fairly large refactoring which meant I needed to completely rework my patch.</p>
<p>The refactoring did actually make my updated patch much simpler though and I have now got it all written, complete with test coverage (my first go at using <a href="http://www.icegreen.com/greenmail/">GreenMail</a> - which I quite like).</p>
<p>This time around my patch adds an envelopeFrom method to the sendMail closure...</p>
<pre><code>sendMail {
envelopeFrom 'verp-fred=smith.com@example.com'
from 'abc@example.com'
to 'fred@smith.com'
subject 'Important email'
body ...
}
</code></pre>
<p>You can find <a href="https://github.com/spither/grails-mail">my version of the grails mail plugin</a> forked on github and hopefully it will be <a href="https://github.com/gpc/grails-mail/pull/3">pulled</a> into the main plugin soon.</p>
<p>It's already in production and working well at <a href="http://www.supajam.com/">SupaJam</a> (sending about 50,000 emails per week).</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=email">email</a>
<a rel="tag" href="/simon/blog/?tag=smtp">smtp</a>
<a rel="tag" href="/simon/blog/?tag=envelope-sender">envelope sender</a>
<a rel="tag" href="/simon/blog/?tag=verp">verp</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
Grails multi-version launch script2011-03-13T00:00:00+00:002015-01-30T21:50:56+00:00Simon Pitherhttps://www.pither.com/simon/blog/2011/03/13/grails-multi-version-launch-script<article>
<h1>Grails multi-version launch script</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2011-03-13T00:00:00+00:00">Sun 13th Mar 2011</time>
(updated <time datetime="2015-01-30T21:50:56+00:00">Fri 30th Jan 2015</time>)
</div>
<div>
<p><em>Note:</em> The <a href="https://github.com/spither/grails-multi-launch">latest version of the script</a> below is on github.</p>
<p>I work with <a href="http://grails.org/">grails</a> quite a lot these days. I have about 5 plugins and over 10 projects that I currently work on frequently. Fairly inevitably they use an assortment of grails versions. Manually chopping and changing command lines to execute different versions gets pretty tedious.</p>
<p>The solution - have a grails wrapper script that detects the version of grails defined in the current project and runs the right one.</p>
<p>This wasn't my idea, but when I finally got fed up with manually juggling grails versions I couldn't find where I had read about it previously. So I decided to write my own version.</p>
<p>This relies on putting all of the grails releases that you want to support into /usr/local/grails/ using the directory names that come in the distribution file (ie grails-x.x.x). Then just create /usr/local/bin/grails with the following content:</p>
<pre><code>#!/bin/bash
BASE=/usr/local/grails
APP=application.properties
VER=$1
if [ -d "$BASE/grails-$VER" ]; then
shift
else
if [ -f $APP ]; then
VER=`grep app.grails.version $APP | cut -d= -f2`
if [ ! -d "$BASE/grails-$VER" ]; then
echo Detected version $VER could not be found
exit 1
fi
else
echo No valid version on command line and no $APP file found
exit 1
fi
fi
export GRAILS_HOME=$BASE/grails-$VER
$BASE/grails-$VER/bin/grails $*
</code></pre>
<p>In existing projects you can just run "grails" and it will pick the right version - assuming you have it installed. If you want to override the automatic choice (eg when upgrading the project) you can specify a version as the first parameter:</p>
<pre><code>grails 1.3.7 upgrade
</code></pre>
<p>Likewise when not in a project (eg when creating a new one) you can do the same thing:</p>
<pre><code>grails 1.3.5 create-app
</code></pre>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
Custom (private) DTD in HTTPBuilder XML response2010-11-16T00:00:00+00:002013-07-29T13:51:00+00:00Simon Pitherhttps://www.pither.com/simon/blog/2010/11/16/custom-private-dtd-in-httpbuilder-xml-response<article>
<h1>Custom (private) DTD in HTTPBuilder XML response</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2010-11-16T00:00:00+00:00">Tue 16th Nov 2010</time>
(updated <time datetime="2013-07-29T13:51:00+00:00">Mon 29th Jul 2013</time>)
</div>
<div>
<p>The Groovy <a href="http://groovy.codehaus.org/modules/http-builder/">HTTPBuilder</a> library is extremely useful for interacting with all sorts of things, including remote web services that send back XML responses. In these cases HTTPBuilder will automatically use Groovy's <a href="http://groovy.codehaus.org/Reading+XML+using+Groovy's+XmlSlurper">XmlSlurper</a> to parse the response and give you back a GPathResult to make navigating the XML super easy.</p>
<p>However today I've been trying to parse XML that's returned (from a site I have no control over) with only 'SYSTEM "awkward.dtd"' provided to define the DTD. I do have a copy of the DTD and I'm quite happy for it to be used in the parsing, however with only the filename specified it turned out to be fairly tricky to control how the file was found.</p>
<p>Generally speaking, you can pass a custom <a href="http://download.oracle.com/javase/6/docs/api/org/xml/sax/EntityResolver.html">EntityResolver</a> to XmlSlurper to help it find DTDs. However in this case I don't have direct access to the XmlSlurper instance - HTTPBuilder is doing it for me.</p>
<p><a href="http://groovy.codehaus.org/modules/http-builder/apidocs/groovyx/net/http/HTTPBuilder.html">HTTPBuilder (the class)</a> uses a <a href="http://groovy.codehaus.org/modules/http-builder/apidocs/groovyx/net/http/ParserRegistry.html">ParserRegistry</a> to provide the default XML parsing method. Looking at the <a href="http://groovy.codehaus.org/modules/http-builder/apidocs/src-html/groovyx/net/http/ParserRegistry.html#line.240">source code</a> it seems that HTTPBuilder already provides a custom, catalog based, EntityResolver to XmlSlurper. ParserRegistry also provides a convenient [addCatalog](http://groovy.codehaus.org/modules/http-builder/apidocs/groovyx/net/http/ParserRegistry.html#addCatalog(java.net.URL)) method to allow extra catalog definitions, which are intended to define (or at least help resolve) the location of DTDs. The tricky bit was working out what to put in a new catalog!</p>
<p>Firstly I wanted to get some debug information out, so that I could see more clearly how my DTD was currently being searched for. The <a href="http://xml.apache.org/commons/components/apidocs/resolver/org/apache/xml/resolver/tools/CatalogResolver.html">CatalogResolver</a> and related classes will load a few properties from a CatalogManager.properties file on the classpath. To switch on most debug messages, I only needed one property:</p>
<pre><code>verbosity=5
</code></pre>
<p>I'm doing all of this in a <a href="http://grails.org">Grails</a> application, so I created this file in src/java to get it added to the root of my running application classpath.</p>
<p>This revealed just one method call before the exception...</p>
<pre><code>resolveSystem(file:///home/simon/path/to/webapp/awkward.dtd)
2010-11-16 15:45:53,355 [http-8080-1] ERROR errors.GrailsExceptionResolver - /home/simon/path/to/webapp/awkward.dtd (No such file or directory)
java.io.FileNotFoundException: /home/simon/path/to/webapp/awkward.dtd (No such file or directory)
</code></pre>
<p>So it's already resolved the filename to an absolute path (I could just put the DTD file there, but that wouldn't help for the production environment) and is searching for it as a System ID.</p>
<p>This meant my extra catalog definition would have to manipulate the system ID to get the DTD found. I failed miserably to find any (readable and current) documentation on how to write these catalog files, so I ended up reading the source of the <a href="http://svn.apache.org/repos/asf/xml/commons/tags/xml-commons-resolver-1_2/java/src/org/apache/xml/resolver/readers/OASISXMLCatalogReader.java">SAX parser</a> to figure out what I needed...</p>
<pre><code><?xml version="1.0"?>
<!DOCTYPE catalog PUBLIC "-//OASIS/DTD Entity Resolution XML Catalog V1.0//EN"
"http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd">
<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog" prefer="system" xml:base=".">
<!-- http://xml.apache.org/commons/components/resolver/resolver-article.html -->
<systemSuffix systemIdSuffix="awkward.dtd" uri="awkward.dtd"/>
</catalog>
</code></pre>
<p>The final step was to get this catalog file loaded (again from the classpath via placement in src/java). The catalog resolver used in HTTPBuilder is static, so I only needed to load it once in BootStrap.groovy...</p>
<pre><code>import groovyx.net.http.ParserRegistry
class BootStrap {
def init = { servletContext ->
ParserRegistry.addCatalog(MyClass.getResource('/my-catalog.xml'))
}
}
</code></pre>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=groovy">groovy</a>
<a rel="tag" href="/simon/blog/?tag=xml">xml</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
Envelope senders for Grails mail plugin (Spring Java Mail)2010-06-22T00:00:00+00:002013-07-29T13:54:32+00:00Simon Pitherhttps://www.pither.com/simon/blog/2010/06/22/envelope-senders-for-grails-mail-plugin-spring-java-mail<article>
<h1>Envelope senders for Grails mail plugin (Spring Java Mail)</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2010-06-22T00:00:00+00:00">Tue 22nd Jun 2010</time>
(updated <time datetime="2013-07-29T13:54:32+00:00">Mon 29th Jul 2013</time>)
</div>
<div>
<p>I would have expected the desire to set a specific envelope sender address for mails from a web-app to be fairly common. However finding information on how to do it with the <a href="http://grails.org/plugin/mail">Grails mail plugin</a> (ie Spring mail (wrappers for JavaMail)) proved much harder than I expected.</p>
<p>In fact, I ended up reading the source code!</p>
<p>It turns out that setting a single, application wide envelope sender is pretty easy (and is actually documented in various places). Simply set a value in Config.groovy such as:</p>
<pre><code>grails.mail.smtp.from = "envelope@domain.com"
</code></pre>
<p>In my case though, I was looking to implement <a href="http://cr.yp.to/proto/verp.txt">VERP</a> for a mailing list, so that wasn't suitable.</p>
<p>From reading the source I discovered that the JavaMail SMTPTransport would actually check if the message it was given was an SMTPMessage and would respect the envelopeFrom property if it was.</p>
<p>To take advantage of this but still make use of the message builder from the mail plugin, I wrote a small service that wraps around the normal mailService:</p>
<pre><code>import com.sun.mail.smtp.SMTPMessage
import javax.mail.internet.MimeMessage
import org.springframework.mail.MailMessage
import org.springframework.mail.javamail.MimeMailMessage
import org.grails.mail.MailMessageBuilder
class AdvancedMailService {
static transactional = false
def mailService
def mailSender
MailMessage sendMail(String envelopeFrom, Closure callable) {
def messageBuilder = new MailMessageBuilder(mailService, mailSender)
callable.delegate = messageBuilder
callable.resolveStrategy = Closure.DELEGATE_FIRST
callable.call()
def message = messageBuilder.createMessage()
mailService.initMessage(message)
MimeMessage mimeMsg
if(message instanceof MimeMailMessage)
mimeMsg = message.getMimeMessage()
else
mimeMsg = message
SMTPMessage smtpMsg = new SMTPMessage(mimeMsg)
smtpMsg.envelopeFrom = envelopeFrom
mailService.sendMail(smtpMsg)
return message
}
}
</code></pre>
<p>You have to call it as a specific service, but otherwise it's much the same as the normal mail service, eg:</p>
<pre><code>advancedMailService.sendMail("list-recipient=theirdomain.com@domain.com") {
from 'Mailing List <list@domain.com>'
to "recipient@theirdomain.com"
subject "Test mail"
body(view: "/mails/test1", model: model)
headers([
'Precedence': 'bulk',
])
}
</code></pre>
<p>Now I just need to set something up to process the bounces. I wonder how easy it would be to embed <a href="http://james.apache.org/">James</a>!?</p>
<p>Update: <a href="/articles/2011/03/13/grails-mail-patch-for-envelopefrom-verp">revised approach</a>, complete with patch sent to grails-mail plugin.</p>
<p>Update: I didn't use James, but I did finally <a href="/articles/2011/09/20/add-an-smtp-server-to-a-grails-application">accept SMTP mail in Grails</a>.</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=email">email</a>
<a rel="tag" href="/simon/blog/?tag=smtp">smtp</a>
<a rel="tag" href="/simon/blog/?tag=envelope-sender">envelope sender</a>
<a rel="tag" href="/simon/blog/?tag=spring">spring</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
Dependency excludes in Grails 1.22010-02-01T00:00:00+00:002013-07-29T13:55:48+00:00Simon Pitherhttps://www.pither.com/simon/blog/2010/02/01/dependency-excludes-in-grails-1.2<article>
<h1>Dependency excludes in Grails 1.2</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2010-02-01T00:00:00+00:00">Mon 1st Feb 2010</time>
(updated <time datetime="2013-07-29T13:55:48+00:00">Mon 29th Jul 2013</time>)
</div>
<div>
<p>I ended up reading the Grails code to figure this out, so I thought it might be useful to record.</p>
<p>One of the projects that I recently upgraded to <a href="http://grails.org/">Grails 1.2</a> makes use of <a href="http://groovy.codehaus.org/HTTP+Builder">Groovy HTTP-Builder</a>. If you just pull in HTTP-Builder you'll quickly find that it's dependencies break your grails project. So under 1.1.1 and using maven, I had this:</p>
<pre><code><dependency>
<groupId>org.codehaus.groovy.modules.http-builder</groupId>
<artifactId>http-builder</artifactId>
<version>0.5.0-RC1</version>
<exclusions>
<exclusion>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy</artifactId>
</exclusion>
<exclusion>
<groupId>xml-apis</groupId>
<artifactId>xml-apis</artifactId>
</exclusion>
</exclusions>
</dependency>
</code></pre>
<p>New in Grails 1.2 is <a href="http://grails.org/doc/latest/guide/3.%20Configuration.html#3.7 Dependency Resolution">dependency management</a> via conf/BuildConfig.groovy. There are lots and lots of examples around for how to define dependencies, but significantly less on how to exclude dependencies of those dependencies. So here's mine:</p>
<pre><code>dependencies {
compile('org.codehaus.groovy.modules.http-builder:http-builder:0.5.0-RC1') {
excludes 'groovy', 'xml-apis'
}
}
</code></pre>
<p>The thing that caught me out was that there's no groupId specified on the excludes, only an artifactId.</p>
<p>Something else I noticed while reading the dependency code was that the dependency string matching is very limited. Only three parts (groupId:artifactId:version) can be matched. None of the other (ie type or classifier) maven parameters are supported.</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=dependencies">dependencies</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
Grails 1.2.0 (or 1.2.1 snapshot) upgrade2010-01-30T00:00:00+00:002013-07-29T13:56:53+00:00Simon Pitherhttps://www.pither.com/simon/blog/2010/01/30/grails-1.2.0-or-1.2.1-snapshot-upgrade<article>
<h1>Grails 1.2.0 (or 1.2.1 snapshot) upgrade</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2010-01-30T00:00:00+00:00">Sat 30th Jan 2010</time>
(updated <time datetime="2013-07-29T13:56:53+00:00">Mon 29th Jul 2013</time>)
</div>
<div>
<p>Following the recent release of <a href="http://grails.org/">Grails</a> 1.2.0 I've attempted to upgrade a few of the projects I work on. Some of these attempts have gone better than others.</p>
<p>For one of the simpler ones I only hit two problems.</p>
<ol>
<li>
<p>Combining Acegi with Enum's defined in the same source file as domain objects no longer works. The grails <a href="http://jira.codehaus.org/browse/GRAILSPLUGINS-1575">bug is here</a> and thankfully the workaround is easy - just pull all the Enum's into their own files.</p>
</li>
<li>
<p>Due to some of the performance improvements that Grails 1.2 makes around it's handling of GSPs and Sitemesh, the <a href="http://jira.codehaus.org/browse/GRAILS-5603">addProperty() function no longer works correctly</a></p>
</li>
</ol>
<p>Unfortunately the second problem was not so easy to work around. It does however have a fix committed to the 1.2.x branch of development code. So how do you get that then? Here's what I did:</p>
<pre><code>$ git clone git://github.com/grails/grails.git grails
$ cd grails
$ git checkout origin/1.2.x
</code></pre>
<p>You'll get some warnings about not being on a branch, but assuming you just want to build it and not modify it, you can safely ignore them.</p>
<pre><code>$ ant package
</code></pre>
<p>You'll obviously need ant and a suitable java compiler ready to go before running this last step.</p>
<p>After a couple of minutes you should have a dist/grails-1.2.1.SNAPSHOT.zip file which you can unzip/install in just the same way you would a download from the grails site.</p>
<p>Only thing left after that is to "grails upgrade" your project again to set it to the 1.2.1.SNAPSHOT version!</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
HTML email from a GSP with the Grails mail plugin2009-09-04T00:00:00+00:002013-10-10T11:31:15+00:00Simon Pitherhttps://www.pither.com/simon/blog/2009/09/04/html-email-from-a-gsp-with-the-grails-mail-plugin<article>
<h1>HTML email from a GSP with the Grails mail plugin</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2009-09-04T00:00:00+00:00">Fri 4th Sep 2009</time>
(updated <time datetime="2013-10-10T11:31:15+00:00">Thu 10th Oct 2013</time>)
</div>
<div>
<p>The <a href="http://grails.org/plugin/mail">Grails mail plugin</a> is an invaluable addition to most Grails based sites - it's a rare website that doesn't need to send email. Some really useful features have been sitting on it's todo list for a while, but it still does a lot as it is.</p>
<p>(Examples below are mostly stolen from the <a href="http://grails.org/plugin/mail">plugin documentation</a>.)</p>
<p>It lets you send a string as a plain text email:</p>
<pre><code>sendMail {
to "fred@g2one.com"
subject "Hello Fred"
body 'How are you?'
}
</code></pre>
<p>Or send a string as an HTML email:</p>
<pre><code>sendMail {
to "fred@g2one.com"
subject "Hello John"
html '<b>Hello</b> World'
}
</code></pre>
<p>Then it gets really useful and lets you render your email content from a GSP:</p>
<pre><code>sendMail {
to "john@g2one.com"
subject "Hello John"
body(view:"/mail/confirmationRequest", model:[username:'john'])
}
</code></pre>
<p>This is especially handy when you want to send your mail from a service class or a Quartz job (as these places don't have easy access to the normal rendering methods).</p>
<p>The slight flaw with this is that the <code>body()</code> method defaults to creating a plain text email (contrary to what the documentation says - I'd fix it if the grails.org password reminder page accepted email addresses instead of usernames!) and there is no <code>html(view:"...", model:[])</code> equivalent as there is for simple string parameters.</p>
<p>Instead for GSP templates, the <code>body()</code> method uses information embedded in the GSP document itself to decide if the email should be plain text or HTML. So to produce an HTML email, you need to add this to your GSP:</p>
<pre><code><%@ page contentType="text/html" %>
</code></pre>
<p>As I said above, the mail plugin documentation does talk about this, but it also says that HTML is the default, which appears (from reading the code and testing) not to be the case. You also need to be quite specific with the <code>contentType</code> string to get it to detect as HTML.</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=html">html</a>
<a rel="tag" href="/simon/blog/?tag=email">email</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>
Upgrading a Maven Grails project from 1.1 to 1.1.12009-06-24T00:00:00+00:002013-07-29T13:59:00+00:00Simon Pitherhttps://www.pither.com/simon/blog/2009/06/24/upgrading-a-maven-grails-project-from-1-1-to-1-1-1<article>
<h1>Upgrading a Maven Grails project from 1.1 to 1.1.1</h1>
<div class="article-meta">
Posted
by <span>Simon Pither</span>
on <time datetime="2009-06-24T00:00:00+00:00">Wed 24th Jun 2009</time>
(updated <time datetime="2013-07-29T13:59:00+00:00">Mon 29th Jul 2013</time>)
</div>
<div>
<p>Grails 1.1.1 has been out for a while now. For those people who use grails directly the upgrade path is pretty simple (download, install, run "grails upgrade"). However for those of us who haven decided to build our projects via maven, the path seems to be much less clear.</p>
<p>There is a <a href="http://www.nabble.com/Grails-1.1.1-upgrade---with-maven-in-the-mix-to23544841.html">large mailing list thread</a> or a more <a href="http://grails.org/Maven+Integration#comment_525">succinct comment</a> that do cover what is needed to <em>compile</em> with 1.1.1 for most projects.</p>
<p>Here is that comment repeated in "svn diff" form for the project I've just upgraded:</p>
<pre><code>Index: pom.xml
===================================================================
--- pom.xml (revision 95)
+++ pom.xml (working copy)
@@ -10,12 +10,12 @@
<dependency>
<groupId>org.grails</groupId>
<artifactId>grails-crud</artifactId>
- <version>1.1</version>
+ <version>1.1.1</version>
</dependency>
<dependency>
<groupId>org.grails</groupId>
<artifactId>grails-gorm</artifactId>
- <version>1.1</version>
+ <version>1.1.1</version>
</dependency>
<!-- Grails defaults to OSCache for the second-level Hibernate cache. -->
@@ -58,6 +58,14 @@
<type>jar</type>
<scope>compile</scope>
</dependency>
+
+ <!-- This is really a Grails 1.1.1 dependency, but it's not yet declared as such, so the work around is to include it here... -->
+ <dependency>
+ <groupId>org.tmatesoft.svnkit</groupId>
+ <artifactId>svnkit</artifactId>
+ <version>1.2.3.5521</version>
+ <scope>runtime</scope>
+ </dependency>
</dependencies>
<repositories>
@@ -99,7 +107,7 @@
<plugin>
<groupId>org.grails</groupId>
<artifactId>grails-maven-plugin</artifactId>
- <version>1.0</version>
+ <version>1.1-SNAPSHOT</version>
<extensions>true</extensions>
<executions>
<execution>
Index: application.properties
===================================================================
--- application.properties (revision 97)
+++ application.properties (working copy)
@@ -9,4 +9,4 @@
app.servlet.version=2.4
plugins.hibernate=1.1.1
app.name=webapp
-app.grails.version=1.1
+app.grails.version=1.1.1
</code></pre>
<p>Unfortunately that's not the whole storey though, as there are still several files that live within your project tree that are really part of Grails, and have changed between 1.1 and 1.1.1. So you have to put your project through the "grails upgrade" process to pick them up.</p>
<p>In theory running <code>mvn grails:exec -Dcommand="upgrade"</code> should do the job. But it doesn't. It managed about half of the job and then fails with an error:</p>
<pre><code>Embedded error: java.lang.reflect.InvocationTargetException
/home/simon/build/cosalt/trunk/webapp/null/src/war not found.
</code></pre>
<p>So the solution seems to require a download of <a href="http://grails.org/download/file?mirror=99">Grails 1.1.1</a>...</p>
<pre><code>$ cd /tmp
$ wget http://grails.org/download/file?mirror=99
$ tar -xzf grails-bin-1.1.1.tar.gz
$ cd <your project dir>
$ GRAILS_HOME=/tmp/grails-1.1.1 /tmp/grails-1.1.1/bin/grails upgrade
<confirm the upgrade>
</code></pre>
<p>That will add and replace a few files within your project tree. To add the new ones to svn:</p>
<pre><code>$ svn add ivysettings.xml ivy.xml webapp-test.launch
</code></pre>
<p>One of the files that gets replaced is .classpath, but it gets replaced with lots of things that really aren't suitable for a maven project, so:</p>
<pre><code>$ svn revert .classpath
</code></pre>
<p>Commit it all and you should be fully upgraded to 1.1.1. I do hope the Grails developers improve this process a little in the future!</p>
<p>While upgraded, that still wasn't the whole story for this particular project as it also uses the quartz plugin which <a href="http://jira.codehaus.org/browse/GRAILSPLUGINS-1159">disagrees with Grails 1.1.1</a> (whether you use maven or not). There is an easy work around available though, assuming you don't want to run quartz jobs in your integration tests. Here's the change I needed in svn diff format again:</p>
<pre><code>Index: grails-app/conf/QuartzConfig.groovy
===================================================================
--- grails-app/conf/QuartzConfig.groovy (revision 95)
+++ grails-app/conf/QuartzConfig.groovy (working copy)
@@ -1,6 +1,11 @@
//
//
quartz {
- autoStartup = true
+ if (Environment.current != Environment.TEST) {
+ autoStartup = true
+ }
+ else {
+ autoStartup = false
+ }
jdbcStore = false
}
</code></pre>
<p>My project has gained a couple of warnings, but still passes it's tests and seems to work, in theory now with <a href="http://grails.org/1.1.1+Release+Notes">less bugs and 30% faster GSP rendering</a>!</p>
</div>
<div class="tags-panel panel panel-default">
<div class="panel-body">
Tags:
<a rel="tag" href="/simon/blog/?tag=grails">grails</a>
<a rel="tag" href="/simon/blog/?tag=maven">maven</a>
<a rel="tag" href="/simon/blog/?tag=upgrade">upgrade</a>
<a rel="tag" href="/simon/blog/?tag=1.1.1">1.1.1</a>
<a rel="tag" href="/simon/blog/?tag=technology">technology</a>
</div>
</div>
</article>