[Anthill-pro] Tighter Maven Integration

[Jason Dillon]

On Mar 16, 2007, at 8:07 AM, Jason Schatz wrote:
> When you described the solution you took for your projects you said  
> that
> you started of with a fresh repository for each build as well as each
> dependency.

Yes... I have AHP clean the project workspace each time then set the  
repo directory to ./repository, so its always empty when the AHP  
build starts.  And I do have AHP setup to install any dependency AH  
project's Maven Repository artifact set into ./repository to populate  
the dependencies output (as well as anything it depended upon).


> Then when you build each dependency you copy the full
> contents of its repository into the current builds.

Yes.


> I was just wondering your reasoning for taking this approach, was  
> it to
> cover the case where project A depends on project B and project B gets
> built with version 15 of axis and now project A is going to be built
> with version 16 of axis?

There are a few reasons.  First, more often than not, if project A  
depends on some external artifact, and project B depends on A...  
there is a good chance that B also has a dependency on that external  
artifact.

Since the build for project A already has acquired the ext artifact,  
it makes sense to deliver the artifact from A's build to B's build  
and avoid any complications due to B's downloading the artifact again  
from remote repositories.  I have seen some failures where A could  
download the artifact, but B failed due to some network issue, or  
maybe because the repo serving it had a maintenance issue or faulty  
rewrite rules, etc.

So, delivering the artifacts from A to B helps ensure that B has  
access to *everything* required to build A, and as a side effect has  
*everything* which A might depend upon transitively.

This also reduces network traffic to remote repositories as well.

The second major reason is to handle cases when SNAPSHOT artifacts  
are used.  This is very similar to the situation above, except that  
Maven has special handling for artifacts with "SNAPSHOT" in the  
version.  BUT, with the current Maven 2.0.x series there is no real  
gain... but I did get a patch applied to Maven 2.1 which will disable  
any SNAPSHOT updates... which means that Maven will continue to  
download artifacts which are not in the local repository just like it  
does normally when online... but it will not try to do any SNAPSHOT  
updates.  So, if a SNAPSHOT artifact is in the local repo, it will  
just use it and not try to see if there are newer ones (and ignores  
any configured policy, like check once a day or something).

This is very important, as many projects to setup SNAPSHOTs to update  
once a day... and if you have project A building at 11:40pm, and then  
project B building at 12:10am, w/o this mechanism it is entirely  
possible that A and B will end up using *different* artifacts for the  
same version of a dependency.

Anyways... so as mentioned for Maven 2.0.x this scenario will happen,  
but for Maven 2.1.x you can specify -nsu (I think that was the flag,  
no-snapshot-updates).  Not sure if this will make it into 2.0.x,  
though I imagine its easy enough to do... I might patch that as well.

And the last major reason for soaking up the entire local repo  
cache... is that there is _no good way_ to figure out what artifacts  
a project actually produced (instead of downloaded as a dependency).   
It is possible to setup a distributionManagement configuration for  
each project and then use `mvn deploy` which will publish only the  
generated artifacts... BUT that requires changes to the project's  
source (pom configuration) which isn't always possible.  Say for  
example if you wanted to build Apache Geronimo... you don't have  
access to update the pom.xml to add the required configuration to  
make `mvn deploy` work.  That... and `mvn deploy` is horribly slow if  
you use any of the remote wagon methods.  Its reasonably fast when  
using the file-based wagon though.

IMO another significant drawback to using deploy here, is you can't  
really just take that entire directory of artifacts and plup it into  
a dependent projects repo cache and expect it to work.  This is  
primarily due to the representation of SNAPSHOT artifacts in the  
local repo vs remote repo.  Except for a few buggy situations,  
SNAPSHOT artifacts are published with "<version>-<timestamp>- 
<buildnumber>" versions in remote repos, and downloaded as "<version>- 
SNAPSHOT" into the local cache.  This isn't *always* the case...  
unfortunately as that may be... but I believe its due to some bugys  
floating around inside of Maven (as well as some misuse of $ 
{pom.version} in poms on modules which are deployed as snaps).


> I know that maven2, like anthill3, handles transitive dependencies,  
> so I
> was just wondering if this was the sort of scenario you were trying to
> cover in your approach?

Yes, Maven2 does handle transitive dependencies. There are situations  
where 2 (or more) builds may need to depend upon the same artifact  
and they end up actually using 2 different versions, or dependent  
builds fail to download, where dependency build resolved artifacts fine.

It does not happen often... but it does happen... where someone re- 
releases a specific version of an artifact.  Say for example I have  
project A depending on project B, both depend on foo/1.0/ 
foo-1.0.jar.  I publish foo-1.0.jar (1).  Then project A starts to  
build and picks up (1), then I realizes that I messed something up  
and I publish foo-1.0.jar (2) with the fix.  Then project A finishes  
its build with (1) and project B starts up... and its going to get  
(2), not (1).

This is not common, but does happen from time to time.  By shipping  
(1) from A's build to B... the build of B will be predictable wrt the  
relationship to foo-1.0.jar.

It may be that (1) and (2) are both compatible and you won't see any  
difference, but if say (1) was missing some class, or compiled with  
the wrong jdk, then you would see signification differences between  
(1) and (2).

So, by soaking up the entire local repo cache, delivering it to  
dependencies, I can eliminate that problem altogether.

  * * *

Hope this helps... let me know if you would like any more  
clarification, I'm glad to offer it to you guys :-)

--jason