Quantcast

Distributing projects dependencies using p2 and Hudson

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Distributing projects dependencies using p2 and Hudson

jserup1234
We have a number of separate projects containing multiple eclipse plugins, fragments and features that we build with maven3/tycho.

As an example project A contains 15 eclipse plugins, a parent project/pom, a target project and a single feature which includes the plugins.

Each time a developer commits code to project A hudson builds the plugins and build a "deployable" feature which we copy to tomcat using maven-antrun. This feature works as a p2 site that other projects can use/refer to.

Now another project B, much similar in structure to project A, has a dependency to some of the plugins in project A. Therefore we specify in the target project of B that it should point to the feature/p2 site generated when project A is build. To makes this easier we use composite p2 sites (compositeContent.xml and compositeArtifacts.xml).

Currently the above setup works but it could be interesting to hear what you guys do and maybe get some tips based on the below questions:

1) Currently tycho does not seem to have good support for distributing dependencies between separate projects so how do you deal with this? Do you simply avoid creating separate projects and have a single parent pom for all subprojects (plugins, fragments, features etc)?

2) On average a project (around 15-20 plugins) takes approx 3 minutes to build on hudson (we have even made a mirror of the public eclipse update sites), and typically triggers another project to be build if it has a dependency on the project just build. Now this results in a large number of projects being build even if the committed change is small. How do you guys handle integration builds when the number of separate interdependent projects grow?

3) We used to use the FeaturesAndBundlesPublisher to append the updated projects to the existing p2 site but since that required having an eclipse installation on the server and setting up some messy scripts we no longer do this. As a consequence its only possible to get the "latest" version of a project since the p2 sites are rebuild each time a developer commits code. How do you handle latest vs versioned projects?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Distributing projects dependencies using p2 and Hudson

ejain
On Thu, Apr 7, 2011 at 05:32, jserup1234 <[hidden email]> wrote:
> 1) Currently tycho does not seem to have good support for distributing
> dependencies between separate projects so how do you deal with this? Do you
> simply avoid creating separate projects and have a single parent pom for all
> subprojects (plugins, fragments, features etc)?

That's what we do right now. One issue with this approach is that each
small change causes the clients to update all their plugins and
features. This approach also won't work too well once we need to start
supporting multiple versions of certain features. What I'd really like
to do is have the build server build each plugin (or feature or
product) on its own. If I then make a change to plugin A, the build
server should be smart enough to also rebuild features that include
this plugin and all products that include any of these features.
Bamboo has a feature to figure out dependencies of Maven projects, but
I haven't tested it to see if it works with Tycho yet. Does this work
with Hudson/Jenkins? How do you deal with fragments and fragments
containing tests? I'd be quite interested to hear if anyone got this
all figured out...


> 3) We used to use the FeaturesAndBundlesPublisher to append the updated
> projects to the existing p2 site but since that required having an eclipse
> installation on the server and setting up some messy scripts we no longer do
> this. As a consequence its only possible to get the "latest" version of a
> project since the p2 sites are rebuild each time a developer commits code.
> How do you handle latest vs versioned projects?

You can run the FeaturesAndBundlesPublisher like so:

  https://docs.sonatype.org/display/TYCHO/Tycho-extras+-+FeaturesAndBundlesPublisher
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Distributing projects dependencies using p2 and Hudson

Tobias Oberlies
In reply to this post by jserup1234
Please use the new mailing list [hidden email] for future questions.


jserup1234 wrote:
> 2) On average a project (around 15-20 plugins) takes approx 3 minutes to
> build on hudson (we have even made a mirror of the public eclipse update
> sites), and typically triggers another project to be build if it has a
> dependency on the project just build. Now this results in a large number
> of
> projects being build even if the committed change is small. How do you
> guys
> handle integration builds when the number of separate interdependent
> projects grow?

Why do you need to build independent projects on commit? If they are independent, you won't need to build them. If they anyway have SNAPSHOT dependencies, you may consider putting them into the same reactor.
Not sure this answers your question though...

Regards
Tobias

 
> --
> View this message in context:
> http://software.2206966.n2.nabble.com/Distributing-projects-dependencies-
> using-p2-and-Hudson-tp6249782p6249782.html
> Sent from the Tycho Users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Distributing projects dependencies using p2 and Hudson

ejain
On Tue, Apr 19, 2011 at 02:15, Oberlies, Tobias <[hidden email]> wrote:
> Why do you need to build independent projects on commit? If they are independent, you won't need to build them. If they anyway have SNAPSHOT dependencies, you may consider putting them into the same reactor.

Let's say I have features f1 and f2. f1 includes b1, b2 and b3. f2
includes b3, b4 and b5. If I commit a change to b5, I only want b5 and
f2 to be re-built and published. But in our current single-reactor
Tycho setup, every single feature and bundle is rebuilt (and in
consequence downloaded again when a client updates itself). I suppose
you could set up separate projects for each feature and bundle on the
build server and let the build server know about all the dependencies,
but that's tedious for projects with more than a few bundles, and I'm
not even sure it would work (e.g. can a fragment be built on its
own?).

Another problem occurs when f1 includes version [1.0,2.0) of b1 and f2
includes version [2.0,3.0) of b1. We'd need to set up two projects for
b1 (b1v1 and b1v2) on the build server and point them to different
branches in the repository. Projects that build bundles or features
that depend on b1 would need to be set up to depend on either b1v1 or
b1v2. Eclipse's headless build has a concept of "mapping" files to
handle this; does Tycho have plans to provide something similar but
with less ugliness?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Distributing projects dependencies using p2 and Hudson

Tobias Oberlies
> -----Original Message-----
> From: Eric Jain [mailto:[hidden email]]
> Sent: 19 April 2011 19:19
> To: [hidden email]
> Subject: Re: [Tycho Users] Distributing projects dependencies using p2 and
> Hudson
>
> On Tue, Apr 19, 2011 at 02:15, Oberlies, Tobias <[hidden email]>
> wrote:
> > Why do you need to build independent projects on commit? If they are
> independent, you won't need to build them. If they anyway have SNAPSHOT
> dependencies, you may consider putting them into the same reactor.
>
> Let's say I have features f1 and f2. f1 includes b1, b2 and b3. f2
> includes b3, b4 and b5. If I commit a change to b5, I only want b5 and
> f2 to be re-built and published. But in our current single-reactor
> Tycho setup, every single feature and bundle is rebuilt (and in
> consequence downloaded again when a client updates itself). I suppose
> you could set up separate projects for each feature and bundle on the
> build server and let the build server know about all the dependencies,
> but that's tedious for projects with more than a few bundles, and I'm
> not even sure it would work (e.g. can a fragment be built on its
> own?).

I think the Maven integration for Hudson/Jenkins does that for Maven projects. So obviously this wouldn't work for Tycho projects out of the box, but Sonatype is planning a new Maven integration for Hudson. You should check if their ideas would cover your use case.

It's not really interesting for me, because my projects build sufficiently fast so that I building the entire reactor works fine for me.
 
> Another problem occurs when f1 includes version [1.0,2.0) of b1 and f2
> includes version [2.0,3.0) of b1. We'd need to set up two projects for
> b1 (b1v1 and b1v2) on the build server and point them to different
> branches in the repository. Projects that build bundles or features
> that depend on b1 would need to be set up to depend on either b1v1 or
> b1v2. Eclipse's headless build has a concept of "mapping" files to
> handle this; does Tycho have plans to provide something similar but
> with less ugliness?

I don't know map files at all, so I am not sure. I haven't heard of anything like that, so probably this means that there are no plans.

Regards
Tobias

Loading...