Bundles with nested .jars?

[Miles Sabin]

Hi folks,

How is this supported currently?

I'm attempting to use the maven-scala-plugin to build a project (the
Scala IDE for Eclipse) which depends on several Eclipse bundles which
contain nested .jars (specifically, org.eclipse.jdt.debug). These
aren't being unpacked, so the outer .jar's path is being passed to the
Scala compiler as a classpath component, and the build then fails (in
exactly the same way that javac would if an unexploded bundle with
nested .jars were fed to it).

Am I right in thinking that the Tycho OSGi compiler handles unpacking
of nested .jars itself, and hence that there's no direct support for
passing expanded classpaths to other tools?

If that's the case, is there an ETA for getting this functionality into Tycho?

And in the meantime, does anyone have a workaround for me?

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Sievers, Jan]

I guess you should start digging here

http://github.com/sonatype/sonatype-tycho/blob/master/tycho-osgi-components/src/main/java/org/codehaus/tycho/BundleProject.java

OsgiBundleProject has a method

    public List<ClasspathEntry> getClasspath( MavenProject project )
    {
        List<ClasspathEntry> classpath =
            (List<ClasspathEntry>) project.getContextValue( TychoConstants.CTX_ECLIPSE_PLUGIN_CLASSPATH );
        if ( classpath == null )
        {
            throw new IllegalStateException();
        }
        return classpath;
    }


which should provide the nested jars as extracted already.
This is what AbstractOsgiCompilerMojo is using.

Regards
Jan

-----Original Message-----
From: Miles Sabin [mailto:[hidden email]]
Sent: Mittwoch, 9. Juni 2010 11:48
To: [hidden email]
Subject: [Tycho Users] Bundles with nested .jars?

Hi folks,

How is this supported currently?

I'm attempting to use the maven-scala-plugin to build a project (the
Scala IDE for Eclipse) which depends on several Eclipse bundles which
contain nested .jars (specifically, org.eclipse.jdt.debug). These
aren't being unpacked, so the outer .jar's path is being passed to the
Scala compiler as a classpath component, and the build then fails (in
exactly the same way that javac would if an unexploded bundle with
nested .jars were fed to it).

Am I right in thinking that the Tycho OSGi compiler handles unpacking
of nested .jars itself, and hence that there's no direct support for
passing expanded classpaths to other tools?

If that's the case, is there an ETA for getting this functionality into Tycho?

And in the meantime, does anyone have a workaround for me?

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Miles Sabin]

On Wed, Jun 9, 2010 at 12:04 PM, Sievers, Jan <[hidden email]> wrote:

> I guess you should start digging here
>
> http://github.com/sonatype/sonatype-tycho/blob/master/tycho-osgi-components/src/main/java/org/codehaus/tycho/BundleProject.java
>
> OsgiBundleProject has a method
>
>    public List<ClasspathEntry> getClasspath( MavenProject project )
>    {
>        List<ClasspathEntry> classpath =
>            (List<ClasspathEntry>) project.getContextValue( TychoConstants.CTX_ECLIPSE_PLUGIN_CLASSPATH );
>        if ( classpath == null )
>        {
>            throw new IllegalStateException();
>        }
>        return classpath;
>    }
>
>
> which should provide the nested jars as extracted already.
> This is what AbstractOsgiCompilerMojo is using.

Thanks for the pointer. I could do with a little clarification ... are
you implying that all tools that need the fully expanded classpath
need to have special support added to them to work with Tycho?

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Miles Sabin]

On Wed, Jun 9, 2010 at 12:04 PM, Sievers, Jan <[hidden email]> wrote:

> I guess you should start digging here
>
> http://github.com/sonatype/sonatype-tycho/blob/master/tycho-osgi-components/src/main/java/org/codehaus/tycho/BundleProject.java
>
> OsgiBundleProject has a method
>
>    public List<ClasspathEntry> getClasspath( MavenProject project )
>    {
>        List<ClasspathEntry> classpath =
>            (List<ClasspathEntry>) project.getContextValue( TychoConstants.CTX_ECLIPSE_PLUGIN_CLASSPATH );
>        if ( classpath == null )
>        {
>            throw new IllegalStateException();
>        }
>        return classpath;
>    }
>
>
> which should provide the nested jars as extracted already.
> This is what AbstractOsgiCompilerMojo is using.

I'm looking at modifying the maven-scala-plugin to read that value.
Unfortunately ClasspathEntry is a Tycho-specific type, so unless I do
something with reflection it looks like I would have to hard-wire in a
dependency on Tycho. Presumably the same goes for all other tools that
need to do the same thing.

Would it be possible to have an attribute representing the expanded
classpath as a list of java.io.Files?

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Sievers, Jan]

for now I guess you can write a separate adapter mojo that has a dependency on tycho, extracts the classpath and makes it available to other mojos/components which do not depend on tycho.

see Edwin Park's sample code at

http://software.2206966.n2.nabble.com/Accessing-the-classpath-td4758543.html#a4909405

the point about ClasspathEntry is that it also has access rules which respect OSGi package visibility (used by the JDT compiler) , so it's not just a list of files.

@Igor do you think we should also expose a simple list of files representing the classpath in the project context to allow easier integration with non-OSGi specific compilers?

Regards
Jan


-----Original Message-----
From: Miles Sabin [mailto:[hidden email]]
Sent: Mittwoch, 9. Juni 2010 14:50
To: [hidden email]
Subject: Re: [Tycho Users] Bundles with nested .jars?

On Wed, Jun 9, 2010 at 12:04 PM, Sievers, Jan <[hidden email]> wrote:

> I guess you should start digging here
>
> http://github.com/sonatype/sonatype-tycho/blob/master/tycho-osgi-components/src/main/java/org/codehaus/tycho/BundleProject.java
>
> OsgiBundleProject has a method
>
>    public List<ClasspathEntry> getClasspath( MavenProject project )
>    {
>        List<ClasspathEntry> classpath =
>            (List<ClasspathEntry>) project.getContextValue( TychoConstants.CTX_ECLIPSE_PLUGIN_CLASSPATH );
>        if ( classpath == null )
>        {
>            throw new IllegalStateException();
>        }
>        return classpath;
>    }
>
>
> which should provide the nested jars as extracted already.
> This is what AbstractOsgiCompilerMojo is using.

I'm looking at modifying the maven-scala-plugin to read that value.
Unfortunately ClasspathEntry is a Tycho-specific type, so unless I do
something with reflection it looks like I would have to hard-wire in a
dependency on Tycho. Presumably the same goes for all other tools that
need to do the same thing.

Would it be possible to have an attribute representing the expanded
classpath as a list of java.io.Files?

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Sievers, Jan]

I had another look at the tycho compiler sources and one way to integrate a different compiler could be via plexus-compiler-api, by subclassing

org.codehaus.plexus.compiler.AbstractCompiler

and registering it as a component just the way the JDT compiler does it, see

http://github.com/sonatype/sonatype-tycho/blob/master/tycho-compiler-jdt/src/main/java/org/codehaus/tycho/compiler/jdt/JDTCompiler.java 

but with a different hint name.

Then you would provide the compilerID hint as a configuration in your pom, say

      <plugin>
        <groupId>org.sonatype.tycho</groupId>
        <artifactId>maven-osgi-compiler-plugin</artifactId>
        <version>${tycho-version}</version>
        <configuration>
          <compilerId>myCompilerId</compilerId>
        </configuration>
      </plugin>

Best Regards
Jan


-----Original Message-----
From: Sievers, Jan [mailto:[hidden email]]
Sent: Mittwoch, 9. Juni 2010 15:51
To: [hidden email]
Subject: RE: [Tycho Users] Bundles with nested .jars?

for now I guess you can write a separate adapter mojo that has a dependency on tycho, extracts the classpath and makes it available to other mojos/components which do not depend on tycho.

see Edwin Park's sample code at

http://software.2206966.n2.nabble.com/Accessing-the-classpath-td4758543.html#a4909405

the point about ClasspathEntry is that it also has access rules which respect OSGi package visibility (used by the JDT compiler) , so it's not just a list of files.

@Igor do you think we should also expose a simple list of files representing the classpath in the project context to allow easier integration with non-OSGi specific compilers?

Regards
Jan


-----Original Message-----
From: Miles Sabin [mailto:[hidden email]]
Sent: Mittwoch, 9. Juni 2010 14:50
To: [hidden email]
Subject: Re: [Tycho Users] Bundles with nested .jars?

On Wed, Jun 9, 2010 at 12:04 PM, Sievers, Jan <[hidden email]> wrote:

> I guess you should start digging here
>
> http://github.com/sonatype/sonatype-tycho/blob/master/tycho-osgi-components/src/main/java/org/codehaus/tycho/BundleProject.java
>
> OsgiBundleProject has a method
>
>    public List<ClasspathEntry> getClasspath( MavenProject project )
>    {
>        List<ClasspathEntry> classpath =
>            (List<ClasspathEntry>) project.getContextValue( TychoConstants.CTX_ECLIPSE_PLUGIN_CLASSPATH );
>        if ( classpath == null )
>        {
>            throw new IllegalStateException();
>        }
>        return classpath;
>    }
>
>
> which should provide the nested jars as extracted already.
> This is what AbstractOsgiCompilerMojo is using.

I'm looking at modifying the maven-scala-plugin to read that value.
Unfortunately ClasspathEntry is a Tycho-specific type, so unless I do
something with reflection it looks like I would have to hard-wire in a
dependency on Tycho. Presumably the same goes for all other tools that
need to do the same thing.

Would it be possible to have an attribute representing the expanded
classpath as a list of java.io.Files?

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Jason van Zyl-2]

I think in this case some simplification and dropping the plexus-compiler-api and just using JDT directly may be better in the long run. It's not like any other compiler can obey OSGi rules anyway.

On Jun 15, 2010, at 9:05 AM, Sievers, Jan wrote:

I had another look at the tycho compiler sources and one way to integrate a different compiler could be via plexus-compiler-api, by subclassing

org.codehaus.plexus.compiler.AbstractCompiler

and registering it as a component just the way the JDT compiler does it, see

http://github.com/sonatype/sonatype-tycho/blob/master/tycho-compiler-jdt/src/main/java/org/codehaus/tycho/compiler/jdt/JDTCompiler.java

but with a different hint name.

Then you would provide the compilerID hint as a configuration in your pom, say

     <plugin>
       <groupId>org.sonatype.tycho</groupId>
       <artifactId>maven-osgi-compiler-plugin</artifactId>
       <version>${tycho-version}</version>
       <configuration>
         <compilerId>myCompilerId</compilerId>
       </configuration>
     </plugin>

Best Regards
Jan


-----Original Message-----
From: Sievers, Jan [mailto:[hidden email]]
Sent: Mittwoch, 9. Juni 2010 15:51
To: [hidden email]
Subject: RE: [Tycho Users] Bundles with nested .jars?

for now I guess you can write a separate adapter mojo that has a dependency on tycho, extracts the classpath and makes it available to other mojos/components which do not depend on tycho.

see Edwin Park's sample code at

http://software.2206966.n2.nabble.com/Accessing-the-classpath-td4758543.html#a4909405

the point about ClasspathEntry is that it also has access rules which respect OSGi package visibility (used by the JDT compiler) , so it's not just a list of files.

@Igor do you think we should also expose a simple list of files representing the classpath in the project context to allow easier integration with non-OSGi specific compilers?

Regards
Jan


-----Original Message-----
From: Miles Sabin [mailto:[hidden email]]
Sent: Mittwoch, 9. Juni 2010 14:50
To: [hidden email]
Subject: Re: [Tycho Users] Bundles with nested .jars?

On Wed, Jun 9, 2010 at 12:04 PM, Sievers, Jan <[hidden email]> wrote:

I guess you should start digging here

http://github.com/sonatype/sonatype-tycho/blob/master/tycho-osgi-components/src/main/java/org/codehaus/tycho/BundleProject.java

OsgiBundleProject has a method

   public List<ClasspathEntry> getClasspath( MavenProject project )

   {

       List<ClasspathEntry> classpath =

           (List<ClasspathEntry>) project.getContextValue( TychoConstants.CTX_ECLIPSE_PLUGIN_CLASSPATH );

       if ( classpath == null )

       {

           throw new IllegalStateException();

       }

       return classpath;

   }

which should provide the nested jars as extracted already.

This is what AbstractOsgiCompilerMojo is using.

I'm looking at modifying the maven-scala-plugin to read that value.
Unfortunately ClasspathEntry is a Tycho-specific type, so unless I do
something with reflection it looks like I would have to hard-wire in a
dependency on Tycho. Presumably the same goes for all other tools that
need to do the same thing.

Would it be possible to have an attribute representing the expanded
classpath as a list of java.io.Files?

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

Thanks,

Jason

----------------------------------------------------------
Jason van Zyl
Founder,  Apache Maven
http://twitter.com/jvanzyl
---------------------------------------------------------

People develop abstractions by generalizing from concrete examples.
Every attempt to determine the correct abstraction on paper without
actually developing a running system is doomed to failure. No one
is that smart. A framework is a resuable design, so you develop it by
looking at the things it is supposed to be a design of. The more examples
you look at, the more general your framework will be.

  -- Ralph Johnson & Don Roberts, Patterns for Evolving Frameworks 

[Miles Sabin]

On Tue, Jun 15, 2010 at 2:21 PM, Jason van Zyl <[hidden email]> wrote:
> I think in this case some simplification and dropping the
> plexus-compiler-api and just using JDT directly may be better in the long
> run. It's not like any other compiler can obey OSGi rules anyway.

I don't really understand this comment ... could you expand a little?

The official build of the Scala compiler doesn't currently obey OSGi
access rules, but I know of at least one group who've modified it to
do so, and I'd like to see their work merged back into the main Scala
distribution at some point.

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Josh Suereth]

Actually, it's relatively simple to start embedding OSGi rules into the scala compiler via compiler plugins.   Most of the work is in parsing OSGi manifests and setting up a correct classpath.   The scala compiler exposes a plugin API that lets you enforce visibility constraints rather simply.  As Miles stated, there is a group that has done this with Tycho.

The real issue with the plexus compiler API, if I recall correctly from a few years ago, is that it did not support Scala's complex file -> classfile dependency mappings.  This meant a lot of stale classfiles.  Is this has changed, perhaps it is a longer term option.

- Josh

On Tue, Jun 15, 2010 at 9:21 AM, Jason van Zyl <[hidden email]> wrote:

I think in this case some simplification and dropping the plexus-compiler-api and just using JDT directly may be better in the long run. It's not like any other compiler can obey OSGi rules anyway.

On Jun 15, 2010, at 9:05 AM, Sievers, Jan wrote:

I had another look at the tycho compiler sources and one way to integrate a different compiler could be via plexus-compiler-api, by subclassing

org.codehaus.plexus.compiler.AbstractCompiler

and registering it as a component just the way the JDT compiler does it, see

http://github.com/sonatype/sonatype-tycho/blob/master/tycho-compiler-jdt/src/main/java/org/codehaus/tycho/compiler/jdt/JDTCompiler.java

but with a different hint name.

Then you would provide the compilerID hint as a configuration in your pom, say

     <plugin>
       <groupId>org.sonatype.tycho</groupId>
       <artifactId>maven-osgi-compiler-plugin</artifactId>
       <version>${tycho-version}</version>
       <configuration>
         <compilerId>myCompilerId</compilerId>
       </configuration>
     </plugin>

Best Regards
Jan


-----Original Message-----
From: Sievers, Jan [mailto:[hidden email]]
Sent: Mittwoch, 9. Juni 2010 15:51
To: [hidden email]
Subject: RE: [Tycho Users] Bundles with nested .jars?

for now I guess you can write a separate adapter mojo that has a dependency on tycho, extracts the classpath and makes it available to other mojos/components which do not depend on tycho.

see Edwin Park's sample code at

http://software.2206966.n2.nabble.com/Accessing-the-classpath-td4758543.html#a4909405

the point about ClasspathEntry is that it also has access rules which respect OSGi package visibility (used by the JDT compiler) , so it's not just a list of files.

@Igor do you think we should also expose a simple list of files representing the classpath in the project context to allow easier integration with non-OSGi specific compilers?

Regards
Jan


-----Original Message-----
From: Miles Sabin [mailto:[hidden email]]
Sent: Mittwoch, 9. Juni 2010 14:50
To: [hidden email]
Subject: Re: [Tycho Users] Bundles with nested .jars?

On Wed, Jun 9, 2010 at 12:04 PM, Sievers, Jan <[hidden email]> wrote:

I guess you should start digging here

http://github.com/sonatype/sonatype-tycho/blob/master/tycho-osgi-components/src/main/java/org/codehaus/tycho/BundleProject.java

OsgiBundleProject has a method

   public List<ClasspathEntry> getClasspath( MavenProject project )

   {

       List<ClasspathEntry> classpath =

           (List<ClasspathEntry>) project.getContextValue( TychoConstants.CTX_ECLIPSE_PLUGIN_CLASSPATH );

       if ( classpath == null )

       {

           throw new IllegalStateException();

       }

       return classpath;

   }

which should provide the nested jars as extracted already.

This is what AbstractOsgiCompilerMojo is using.

I'm looking at modifying the maven-scala-plugin to read that value.
Unfortunately ClasspathEntry is a Tycho-specific type, so unless I do
something with reflection it looks like I would have to hard-wire in a
dependency on Tycho. Presumably the same goes for all other tools that
need to do the same thing.

Would it be possible to have an attribute representing the expanded
classpath as a list of java.io.Files?

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

Thanks,

Jason

----------------------------------------------------------
Jason van Zyl
Founder,  Apache Maven
http://twitter.com/jvanzyl
---------------------------------------------------------

People develop abstractions by generalizing from concrete examples.
Every attempt to determine the correct abstraction on paper without
actually developing a running system is doomed to failure. No one
is that smart. A framework is a resuable design, so you develop it by
looking at the things it is supposed to be a design of. The more examples
you look at, the more general your framework will be.

  -- Ralph Johnson & Don Roberts, Patterns for Evolving Frameworks 

[Miles Sabin]

On Wed, Jun 16, 2010 at 12:19 AM, Josh Suereth <[hidden email]> wrote:
> The real issue with the plexus compiler API, if I recall correctly from a
> few years ago, is that it did not support Scala's complex file -> classfile
> dependency mappings.  This meant a lot of stale classfiles.  Is this has
> changed, perhaps it is a longer term option.

I have a hunch that the work done on scalac's buildmanager, the Scala
Eclipse builder and sbt over the last year or two could help with
that: one of the outputs of scalac's build manager is a simple text
representation of the source file level dependencies and the source
file => class file mappings.

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Josh Suereth]

Yes, we plan to use that in the maven-scala-plugin once it's officially released.   Perhaps we should move the discussion to the maven-and-scala list for that, so David has a chance to chime in.

On Wed, Jun 16, 2010 at 4:50 AM, Miles Sabin <[hidden email]> wrote:

On Wed, Jun 16, 2010 at 12:19 AM, Josh Suereth <[hidden email]> wrote:
> The real issue with the plexus compiler API, if I recall correctly from a
> few years ago, is that it did not support Scala's complex file -> classfile
> dependency mappings.  This meant a lot of stale classfiles.  Is this has
> changed, perhaps it is a longer term option.

I have a hunch that the work done on scalac's buildmanager, the Scala
Eclipse builder and sbt over the last year or two could help with
that: one of the outputs of scalac's build manager is a simple text
representation of the source file level dependencies and the source
file => class file mappings.

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Igor Fedorenko-2]

Sievers, Jan wrote:

> for now I guess you can write a separate adapter mojo that has a
> dependency on tycho, extracts the classpath and makes it available to
> other mojos/components which do not depend on tycho.
>
> see Edwin Park's sample code at
>
> http://software.2206966.n2.nabble.com/Accessing-the-classpath-td4758543.html#a4909405
>
>
> the point about ClasspathEntry is that it also has access rules which
> respect OSGi package visibility (used by the JDT compiler) , so it's
> not just a list of files.
>
> @Igor do you think we should also expose a simple list of files
> representing the classpath in the project context to allow easier
> integration with non-OSGi specific compilers?

I think Tycho should inject full classpath, including any nested jars,
back into Maven project model (without classpath access rules,
obviously). I actually started implementing this at one point, but did
not have time to finish.

--
Regards,
Igor

[Miles Sabin]

On Fri, Jun 18, 2010 at 5:04 AM, Igor Fedorenko <[hidden email]> wrote:
> I think Tycho should inject full classpath, including any nested jars,
> back into Maven project model (without classpath access rules,
> obviously). I actually started implementing this at one point, but did
> not have time to finish.

That would certainly make integration with other tools more straightforward.

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin

[Sievers, Jan]

Hi Miles,

FYI - nested jars and PDE source folders are injected back into maven dependency model now (with minor limitation), see https://issues.sonatype.org/browse/TYCHO-483 
This patch will go into tycho 0.11.0.

Regards
Jan

-----Original Message-----
From: Miles Sabin [mailto:[hidden email]]
Sent: Freitag, 18. Juni 2010 09:51
To: [hidden email]
Subject: Re: [Tycho Users] Bundles with nested .jars?

On Fri, Jun 18, 2010 at 5:04 AM, Igor Fedorenko <[hidden email]> wrote:
> I think Tycho should inject full classpath, including any nested jars,
> back into Maven project model (without classpath access rules,
> obviously). I actually started implementing this at one point, but did
> not have time to finish.

That would certainly make integration with other tools more straightforward.

Cheers,


Miles

--
Miles Sabin
tel: [phone]
gtalk: [hidden email]
skype: milessabin
http://www.chuusai.com/
http://twitter.com/milessabin