GeoTools : 1 November 1st

Conversation with #geotools at 2004-11-01 12:22:20 on (irc)
(12:22:20) : The topic for #geotools is:
(12:24:43) cholmes entered the room.
(12:30:35) jmacgill: hi all
(12:30:49) polio: heya
(12:30:58) jeichar: hi
(12:31:10) jmacgill: ok, its now the meeting time regdless of which time zone you are in
(12:31:48) jmacgill: Agenda itmes:
(12:31:56) jmacgill: GT2.0 update
(12:32:00) polio: we have a geoapi issue
(12:32:18) jmacgill: Home for standard feature types...
(12:32:45) jmacgill: uDIG's rendering architecture, and implications for GeoTools
(12:34:18) polio: GT2.0 update:
(12:36:05) jmacgill: ok, I have a candiate for release
(12:36:23) jmacgill: I want to have some developers test it to make sure that the 'src' release builds on their machine
(12:36:40) jmacgill: also I want to check that the module list is correct
(12:36:49) jmacgill: i.e. nothing missing and nothing in that should not be
(12:37:09) jmacgill: other than that, I just need to tidy up the README and we are good to go
(12:37:13) jeichar: I don't know about what is expected in the module list but I can download the source and make sure it builds.
(12:37:31) jeichar: where can I get 'src'?
(12:37:38) polio: for the module list I would run it past the list
(12:37:50) jmacgill: thats my plan
(12:38:03) polio: (smile)
(12:38:04) jmacgill: anyone know the limit on confluence attachments?
(12:38:21) polio: not here
(12:38:25) jyutzler entered the room.
(12:38:52) jeichar: nope
(12:39:02) jmacgill: lets find out (smile)
(12:41:35) jmacgill: uploading...
(12:42:59) jmacgill:
(12:43:32) jmacgill: note that 7 previous attempts were certified insane (smile)
(12:43:40) polio: lo
(12:43:41) polio: l
(12:45:31) jmacgill: ok, so, download that and give it a shot
(12:45:36) jmacgill: I'll mention it on devel too.
(12:45:43) polio: cool, will do
(12:45:51) jmacgill: ok, next...
(12:45:55) jmacgill: geoapi issue?
(12:47:00) polio: it more of a request ... but I wanted to ask that we treat switching SNAPSHOT instances in the same way we do for any other api change, as they both have the same effect
(12:48:02) jmacgill: Agreed
(12:48:29) jmacgill: so we want a change proposal to the devel list then a simple vote before a new jar is uploaded.
(12:48:39) polio: yes please
(12:48:50) jmacgill: I know Martin needs to update it frequently - perhaps we can let API additions go though automaticaly
(12:49:00) jmacgill: but changes or subtractions require a vote
(12:49:18) jmacgill: that way at least people will be AWARE before code braks.
(12:49:52) jmacgill: or would you prefere ALL snapshot changes to go though the process?
(12:49:59) polio: that makes sense
(12:50:03) jpmeagher entered the room.
(12:50:19) polio: I was just concerned because of broken code both within and outside the geotools code base
(12:50:41) polio: (additions would be fine ... just deletions/deprecations)
(12:50:56) jmacgill: ok, lets go with change/removal to api require vote before SNAPSHOT jar in repository is required
(12:51:15) polio: sounds good here
(12:51:22) jeichar: agreed
(12:52:18) polio: standard FTs?
(12:53:00) jmacgill: ok, this needs a short and a long term solution
(12:53:07) jpmeagher: wow, good timing on my part
(12:53:30) polio: actually we were just waiting for ya (wink)
(12:53:35) polio: just kidding
(12:53:44) jmacgill: now that we have support for FT inheritance we need a place to keep our key base types
(12:54:00) jmacgill: jpmeagher requires an AnnotationType
(12:54:24) jmacgill: I would actulay quite like a PointType, LineType, PolygonType etc as well.
(12:54:38) jpmeagher: As said in my email earlier today, I'd recommend putting them in a sub package of org.geotools.feature
(12:55:24) cholmes: Sounds good to me.
(12:55:29) jpmeagher: I'd actually like the geometry types also as it would give a way to determine what you had other than using the class of the Geometry object
(12:55:37) polio: I can see that, we would also have to move a few other classes there as well (thinking DefaultFeatureType + DefaultFeature)
(12:55:52) jmacgill: org.geotools.feature.simpleTypes ?
(12:56:19) polio: mm, simpleType has an XSI namespace conotation
(12:56:33) jpmeagher: how about just org.geotools.feature.types?
(12:56:52) polio: much better from my perspective
(12:57:06) jmacgill: sounds good to me
(12:57:33) cholmes: +1
(12:57:35) jpmeagher: on second thought, I don't see any other plural package names so how about type instead of types
(12:57:35) polio: I'm guessing this would also house some defaultFeature implementation too?
(12:58:03) jmacgill: one question is the form aththey should take. Actual classes, .ser, .xsd (that can be parsed into types)
(12:58:10) jpmeagher: It builds off the existing DefaultFeature and DefaultFeatureType, so it's not necessary.
(12:58:47) jmacgill: .ser = serialized instance of DefaultFeatureType
(12:59:06) polio: mmm, but FeatureTypes build Features so concievably ....
(12:59:25) polio: I think java classes for the package
(12:59:50) jpmeagher: I think there may still be some classes or interfaces required so things can pragmatically be referenced without using hardcoded strings.
(13:02:01) polio: we seem stalled ...
(13:02:23) jmacgill: jpmeagher for now is AnnotationType a class?
(13:02:32) jpmeagher: yep... I've got the annotation type written as a class now
(13:02:41) jpmeagher: It's all ready to commit
(13:02:44) jmacgill: lets go with that as a starting point
(13:02:53) jmacgill: commit it to ...feature.type
(13:02:58) jpmeagher: OK
(13:03:06) jmacgill: hang on
(13:03:09) jmacgill: any objections (smile)
(13:03:15) polio: none here
(13:03:28) jmacgill: ok I'm +1 for it
(13:03:32) polio: +1
(13:03:53) jmacgill: Chris already said yes, and as this just an addition
(13:03:54) jmacgill: go for it
(13:04:04) jpmeagher: will do
(13:04:32) jmacgill: I may add the geometry types based on that
(13:04:47) jmacgill: and I'd like to modify Shapefile to use them
(13:04:59) jmacgill: would finaly alow us to have a single SLD document for ALL shapeifles
(13:05:06) jmacgill: rahter than one per geometry type
(13:05:19) cholmes: +1 (sorry)
(13:07:48) jmacgill: jpmeagher do you have some candiatates for utility classes too?
(13:08:13) jpmeagher: Nope, so far everything else is just in the VPF code
(13:08:51) jpmeagher: Rendering code will need to be updated to recognize the Annotation Feature Type though
(13:09:04) jpmeagher: As it would just be displayed as a line otherwise
(13:09:40) polio: does that bring us to "uDIG's rendering architecture, and implications for GeoTools"?
(13:09:40) jmacgill: should not be nessasary, if you create a style that understands it
(13:09:49) jmacgill: but yes, would be good to have a default in place
(13:10:02) jmacgill: polio yes it does.
(13:10:44) jeichar: What concerns do we have with regards to uDIG's rendering architecture? Any question?
(13:11:16) jmacgill: I'm intereted in the 'multiple buffers' solution to a stule with more than one FeatureTypeStyle
(13:11:43) jmacgill: I think we should look at moving some of that architecture though to GeoTools itself (e.g. specific rendereres for specific tasks)
(13:12:35) jeichar: 1- multi buffers
(13:13:23) jeichar: that's an idea so that we can render a single layer in parellel.
(13:14:00) jmacgill: is it working? does it composit well? are there memory implications?
(13:14:24) jeichar: there are most certainly memory considerations. currently we have 1 buffer per layer.
(13:14:36) jeichar: this idea could make a single layer have multiple buffers.
(13:15:06) jeichar: we're not sure we like the idea yet, but Lite currently requires 1 iteration through the feature set per feature style.
(13:15:12) jeichar: that's can be really slow.
(13:15:41) jeichar: It is all theory at this point though.
(13:15:55) jpmeagher: How about caching awt Shapes representing the Features in a WeakHashMap?
(13:16:06) polio: also thinking of tree style caching ...
(13:16:21) polio: we are in swt (eclipse) ... so can't use awt
(13:16:24) jpmeagher: This would allow re-rendering (from a zoom...) to occur without having to go back and read data from the DataStore
(13:16:37) jpmeagher: OK then the swt equivalent of a Shape
(13:16:53) jpmeagher: Basically whatever the onscreen representation of the Feature's Geometry is
(13:16:55) jmacgill: the idea is to have a system capable of rendering more than can be stored in memory
(13:16:58) jeichar: Agreed. These sorts of idea are in the works as we speak. Caching is an absolute requirement.
(13:17:26) jeichar: some levels of caching at any rate.
(13:17:27) jpmeagher: That's where WeakHashMaps are handy, when you run out of memory things are automatically unloaded
(13:17:35) jmacgill: I would have gone for a tile based solution to the multiple FTS problem.
(13:17:55) jpmeagher: When dealing with small data sets everything could be done in memory
(13:17:59) jmacgill: (otherwise the weakhashmap is a problem, you just keep populating it and clearing things from it before you need them again)
(13:18:03) IanS: weak maps are unloaded when no references exists, they are not memory sensitive
(13:18:51) jmacgill: IanS, though the clearing 'tends' to be done during GC operations which 'tend' to occure in low memory situations
(13:19:28) jmacgill: but yeah, no guarantees, instade a Least Recently Used stack is more helpfull.
(13:19:33) IanS: true, just clarifying
(13:19:40) jeichar: James: I agree. Tiling is a good starting point for muliple FTSs.
(13:20:00) jeichar: but for zooms/pans etc... caching is important.
(13:20:33) jpmeagher: Tiling will be tough without the random access to data stores (really geospatial queries) unless you do offline processing
(13:21:39) jmacgill: which is why the multi-buffer approach to multiple FTSs was appealing to me. Whats the memory hit like per buffer?
(13:21:58) jmacgill: quite bad I guess, esp if they have alpha channels
(13:22:09) jeichar: What DSs don't support bbox queries? The index shapefile will be required for fast shapefile but WFSs and DBs should return a smaller bbox.
(13:22:19) jeichar: yes it is bad enough.
(13:22:26) jeichar: sorry smaller featureResults
(13:22:48) jpmeagher: VPF supports it by iterating through the features and only returning the features that are inside the box
(13:23:04) jpmeagher: I think most of the others do the same thing
(13:23:05) jmacgill: with the index support on shapefiles performance would be acceptable
(13:23:12) jeichar: ah too bad
(13:23:37) polio: postgis + wfs do bbox server side
(13:24:21) jmacgill: wonder if we could setup and indexed version of pickle
(13:24:52) jmacgill: and generate our own tile cace during the rendering process
(13:25:19) jeichar: is pickle faster than shapefile?
(13:25:25) polio: or use shp with id and geom only ... that's all we really want to cache
(13:25:55) IanS: cant remember, pickle was very close to shapefile at some point
(13:26:02) jmacgill: erm, what about attribute based styles?
(13:26:28) jeichar: we need to support attribute based style.
(13:26:46) polio: yah i guess ... was thinking we could cut alow of storage out though (for example multiple geoms)
(13:26:52) polio: *alot
(13:27:17) jmacgill: I guess we could setup something like the JAI tilecache system
(13:27:48) jmacgill: The other thing I'm interested in is a way to push features to other types of display as they are loaded - e.g. charts ang graphs
(13:28:17) jmacgill: I don't want to itterate multiple times for each graph/chart and map
(13:28:37) jpmeagher: That's a problem I've run into also
(13:29:11) jmacgill: this would imply some form of event mechanism on a loader that 'notified' renderers and charts of data as it is loaded
(13:29:20) IanS: need a multiplexing iterator
(13:29:34) jmacgill: (if the multi buffer FTS sollution was implemetned, I imagine it would need something similar)
(13:29:41) jmacgill: IanS exactly
(13:29:46) jpmeagher: I've gotten around it by creating an offline processor that converts the features into something that allows fast geospatial queries and caching
(13:30:02) jpmeagher: I would have really liked it to have that at the GeoTools level
(13:30:17) jmacgill: whatst the 'something'?
(13:30:44) jpmeagher: basically a serialized quad tree with file pointers to the features
(13:32:05) jmacgill: ah, so a VPF specific spatial index
(13:32:29) jpmeagher: basically, and not a screwed up geometry storage
(13:33:23) jmacgill: IanS was/is pickle flat?
(13:34:11) IanS: cant remember, but I believe it supports nested features
(13:34:31) jmacgill: humm ok.
(13:34:46) jmacgill: ok, I guess my main point is that uDIG has to solve some of these issues
(13:34:55) jeichar: (smile)
(13:35:10) jmacgill: and I'd like us to consider the possiblity of having those changes occure at the GeoTools level
(13:35:10) jeichar: One large issue about integrating parts of udig is that udig uses eclipse's plugin framework.
(13:35:43) jmacgill: yeah I know, but we should be able to factor out some parts... like a multiplexing feature itterator
(13:36:14) jmacgill: or a multi solution renderer (sometimes uses lite, sometimes uses j2e sometimes uses a WMS renderer)
(13:38:27) jeichar: UDIG's current solution is to have a RenderManager that determines what renderer to use. Each renderer plugin provides a Metrics object that provides metrics for a renderer based on the data source.
(13:38:52) jeichar: RenderManager uses the metrics to determine which renderer to use.
(13:39:13) jmacgill: right, so that sounds like a good candiate for migration to GT some day
(13:40:15) jeichar: If we were to do that we'd need to use a plugin framework. would we use one similar to the one used by DataStores?
(13:40:43) jpmeagher: That's probably a little simplistic for a renderer finder
(13:41:08) jpmeagher: It will need some sort of way to indicate what geometry types ( or ancestor feature types) it handles
(13:41:30) jmacgill: the rendererfactories would be able to provide capabilities information
(13:41:33) jpmeagher: rather than just a simple list of things implementing a particular interface
(13:41:59) jeichar: Eclipse has a xml file for each plugin that provides and the extension point provider provides a xml schema that details the information required.
(13:42:06) polio: wouldn't the metrics represent this?
(13:42:26) polio: so a raster renderer would return a 0 score for feature data
(13:43:27) jeichar: the problem with the metrics idea is that all plugins are loaded up just, ie not lazy loading of jars... but wait does geotools lazy load plugins anyways?
(13:44:00) jmacgill: I'm fairly sure it will end up as lazy loads
(13:44:29) jmacgill: actualy, I think it could (but we need to include the package indexes in the jar manifests)
(13:44:33) cholmes: (Hey guys, I'm taking off, as I need to grab dinner, unless anyone needs anything from me)
(13:45:29) jmacgill: humm, no, it will load them all as the 'canProcess' method is called on each one - so an instance is created
(13:46:15) IanS: the instances can be rather flyweight (and cached)
(13:46:17) jeichar: right. so lets leave that for future considerations. what udig is doing is not worrying about lazy loading at the moment and just loading all the metrics objects and essentially calling 'canProcess()' and 'howFast'
(13:46:42) jeichar: basically.
(13:47:29) jeichar: There is also what we refer to ass a "RenderStack" which merges all the images
(13:47:46) jeichar: that could be a candidate for geotools too.
(13:48:47) jeichar: each renderer renders to a image and the render stack merges all the images. This allows some raster manipulation if need be. IE. turning on/off layer visibility/order etc...
(13:49:27) jeichar: Currently it is stupid, 1 draw command for each layer. but the idea is to get JAI to merge them using hardware acceleration. and JAI smarts.
(13:50:58) jmacgill: ok, so my main request is that, to the largest extent possible, these parts of uDIG remain seperable from the eclipse system
(13:51:27) jmacgill: also that we look at the time required to create a spatial indexed version of pickle
(13:52:09) jmacgill: do we have anything left on the adgenda?
(13:52:20) IanS: i can't really offer much time on dev, but i can help with questions (its a bit convoluted but theres a reason)
(13:52:52) jmacgill: brief explanation of reason for convolutedness would be good at some point (smile)
(13:52:54) jeichar: I will do my best to keep the RenderManager and Stack seperate.
(13:53:02) jeichar: (from eclipse)
(13:53:54) jmacgill: thanks!
(13:56:25) jmacgill: We are low on time (at least I am)
(13:57:18) jmacgill: I have one bigish problem that I need to solve now that we have moved from FeatureCollections to FeatureResults
(13:57:18) jeichar: I don't have any more items.
(13:57:24) jmacgill: I'll post to devel
(13:57:51) jmacgill: basic problem is that I used to be able to get the 'collection' that a feature belonged to with feature.getParent()
(13:58:22) jmacgill: that way I could get an itterator on the collection to find answers to questions like 'max, min, avg...'
(13:59:13) jmacgill: (you can put functions like that into an expression in a style, but the functinos have to apply on a feature by feature basis) so you go max(feature.parent()) (well, sort of)
(13:59:23) jmacgill: now, with FeaureResults there is no way to dot htat
(14:00:07) jmacgill: It would be nice if the feature (or its type) has a link back to the 'source' from which it came
(14:00:25) jmacgill: anyway, thats just food for thought, I'll write it up better on devel
(14:01:28) jmacgill: I have to run
(14:01:36) jmacgill: thanks all!
(14:01:39) IanS: bye
(14:01:45) jeichar: bye
(14:01:46) jpmeagher left the room.
(14:01:57) jmacgill: can someone grab logs!
(14:01:58) jeichar left the room (quit: "Leaving").
(14:05:14) cholmes left the room (quit: Read error: 110 (Connection timed out)).
(14:06:50) IanS left the room (quit: "Leaving").