TerraGo is at ESRI Federal User's Conference

TerraGo has got booth 123 at the ESRI Federal User's Conference. CEO Rick Cobb will be there, along with most of our sales crew.

TerraGo Composer for (ArcGIS) Server

TerraGo is announcing a new product today called TerraGo Composer for Server. It works like this: you interactively select an area of interest (AOI). Composer grovels around ArcGIS Server finding files and documents associated with the AOI. You pick the types of information of interest, and Composer creates a GeoPDF index map of the AOI with hyperlinks to the documents you selected.

First ARGON Mapping Satellite Lauched 48 Years Ago Today

The first ARGON mapping satellite was launched 17 February 1961. The ARGON satellites carried film cameras and ejected film canisters. The imagery was declassified in the Clinton administration. The CORONA, ARGON, and LANYARD satellites exposed more that 2 million feet of film between 1959 and 1972. You can get the imagery and more information from USGS.

GeoPDF and GeoPS with Adobe-Style Payload

Here are the companion (Geo)PostScript and GeoPDF maps to those generated for the worked example. It took me less than 10 minutes to map the TerraGo sytle to the Adobe. We'll delve into the details of what I did in a subsequent post.

Isn't that PostScript GeoPS?

The PostScript file from georegistration worked example post contains a full georegisration. If you put some metadata in there to reliably extract that information, well, that PS is GeoPS!

What's up with the parens in the GeoPDF CTM?

In the georegistration worked example post, we calculated a CTM that takes the page coordinates in points into projected coordinates in meters:

[35.28267 0 0 35.28267 205188.64 3207094.8]

In the PostScript and PDF files, there are CTM entries:

[(35.28267) (0) (0) (35.28267) (205188.64) (3207094.8)]

Same numerical values, just the values in CTMs in the files have parentheses around them. What's up with that?

PostScript and PDF share a similar object system that provides a rich set of types. We'll go into the details of the PDF object system in a later post. However, we'll talk about the types in the two version of the CTM. There are three types present: numbers, strings, and an array. Arrays are delimited with the square backets []. Strings are delimited with parens (). There are actually two types of numbers: integer and real. These types were implemented in earlier versions of Acrobat as 32 bit objects. Real numbers used a fixed point scheme that traded precision for range. That range was too small to hold values often used in geodesy. More recent versions of Acrobat used IEEE 754 single-precision floats, the precision of which is not sufficient store values for geodesic calculations. Rather than try to shoehorn a syntatic extension that used native PDF numbers directly, we stashed the values into strings and extracted the values based on whether a string appeared in a numeric context.

Bonus link: David Goldberg's What Every Computer Scientist Should Know about Floating-Point Arithmetic [PostScript].

GeoPDF Map for Worked Example

As promised, I created a PostScript file using the information we calculated in the worked example post. I created a GeoPDF file by running the PostScript file through Ghostscript to create a PostScript file PDF file that looks like this:


PostScript hacking is fine companion to cubicle lunching...

HT Klokan!

Georegistration: A Worked Example

We're going to work through making a quad sheet and georegistering it. The quad is going to be [90W 29N 89W 30N]. If you're wondering why that quad out of all of the possible quads, it's because I know that 90W 30N hits one of my favorite spots: New Orleans. Let's pick a scale for our quad to be 1:100000, and let's give ourselves one inch margins. We're going to use UTM zone 16 WGS 84 as our coordinate reference system, and I'm going to use GEOTRANS to do the calculations. The the table of longitudes (λ), latitudes (ϕ), eastings and northings is:

  λ  ϕ  E(m)    N(m)
-90 29 207729 3211697
-90 30 210590 3322576
-89 30 307085 3320469
-89 29 305179 3209635

The bounding eastings and northings are:

[207729 3209635 307085 3322576]

Taking the difference between the upper right and lower left values and scaling the result leaves us with a rectangle 0.99356 m by 1.12941 m. To put us into PDF world, we're going to convert the meters to points and add 144 points to the result to give us our one inch margins. The resulting media box is (rounded values):

[0 0 2960 3345]

The neatline falls an inch (72 points) inside the page media box at:
[72 72 2888 3273]
What we've done here is aligned the axes of the projected and page coordinate systems. Moreover, we constrain the scales to be constant regardless of direction. This let's us reduce the problem from solving:

|a c H||h|   |x|
|b d V||v| = |y|
|0 0 1||1|   |1|

to

|a 0 H||h|   |x|
|0 a V||v| = |y|
|0 0 1||1|   |1|

The solution is given by:

a = (x1 - x0)/(h1 - h0)
H = x0 - a h0
V = y0 - a v0

Using the values:

x0 = 207729
y0 = 3209635
x1 = 307085
h0 = v0 = 72
h1 = 2888

yields:

a = 35.28267
H = 205188.64
V = 3207094.8

In PDF notation, the full CTM would be:

[35.28267 0 0 35.28267 205188.64 3207094.8]

This matrix defines the relationship between a point on the page and a point in the projected coordinate system. It's inverse takes the projected coordinate system into the page coordinate system:

[0.028343 0.0 0.0 0.028343 -5815.5645 -90897.17]

Using this, we can discover where the corners of the map neatline will fall:

  λ  ϕ  E(m)    N(m)  H(pt) V(pt)  
-90 29 207729 3211697   72   132
-90 30 210590 3322576  153  3274
-89 30 307085 3320469 2888  3215
-89 29 305179 3209635 2834    74

In a later post, we'll draw this map...

Why Two GeoPDF Specs?

An anonymous reader asked

I'm not sure that the community needs two specs. What am I missing?

This is a hot question, but I hope I can cool it off a bit. The community doesn't need two georegistration techniques, I suppose, but it does have two, so as software providers we need to deal with it. And that's only for PDF. How many ways can you georeference, say a TIFF file? Well, there's world file, at least three different combos of GeoTIFF tags, different metadata payloads cooked up by GIS vendor X, etc. DGN? DWG? DOC? XPS? FOO?

What TerraGo has done with releasing the georegistration technique it used to create and have had created hundreds of thousands of GeoPDF files is to document what we did. This is why it will be published as a best practice rather than a OGC-endorsed specification. People have invested heavily in GeoPDF map libraries that use that technique and we're going to make sure that investment is protected. Similarly, GeoPDF files can be created using the extensions that Adobe has cooked up. We can deal with that, too. Ultimately, I hope that the people who ultimately use, enjoy, and profit from GeoPDF maps and imagery neither know nor care about the details of the arrangement of the bits and bytes that make the georegistration -- just rest easy in knowing that GeoPDF is built on open standards.

If you really get down into the details of how the two techniques differ, there are some contexts which one might be preferable to the other -- and vice versa. Neither is fully baked, IMO, so there will be extensions and revisions. However, GeoPDF's situation is much more akin to GeoTIFF's than a VHS v. Betamax. It's more a matter of flavor than variety. From my perspective, both flavors are delicious. From a GeoPDF user's perspective, it all should "just work".

I will be delving into the details of both techniques in subsequent posts.

GeoTIFF CTM in ASCII Art Glory

Did I mention that I love the GeoTIFF spec. Allow me to reproduce for you the ASCII art version of the section on it's CTM provision:

ModelTransformationTag
      Tag  =  34264  (85D8.H) 
      Type =  DOUBLE    
      N    =  16
      Owner: JPL Cartographic Applications Group

This tag may be used to specify the transformation
matrix between the raster space (and its dependent
pixel-value space) and the  (possibly 3D) model space.
If specified, the tag shall have the following
organization:

      ModelTransformationTag = (a,b,c,d,e....m,n,o,p).

where

        model                              image
        coords =          matrix     *     coords

        |-   -|     |-                 -|  |-   -|
        |  X  |     |   a   b   c   d   |  |  I  |
        |     |     |                   |  |     |
        |  Y  |     |   e   f   g   h   |  |  J  |
        |     |  =  |                   |  |     |
        |  Z  |     |   i   j   k   l   |  |  K  |
        |     |     |                   |  |     |
        |  1  |     |   m   n   o   p   |  |  1  |
        |-   -|     |-                 -|  |-   -|

Some of dat!

GeoPDF Map Frames

We've discussed the concept and mechanisms of georegistration -- the alignment or registration of two Cartesian coordinate systems (one of which is a projected coordinate system (PCS)) via a coordinate transformation matrix. We take this explicit relationship between a PDF page and a specific PCS, give it a name, delineate a region on the page for which the relationship is valid, and name a preferred display coordinate system, and roll it all up into what we call a map frame. I don't now remember from whence the name, but this one turned out OK, IMO.

So here are the map frame pieces:

• map frame description or name
  
• map coordinate system
  
• preferred display coordinate system
  
• page area of map (bounded by neatline)
  

You'll notice that there isn't anything particular to PDF in there. GeoPDF georegistration and map frames, although talked about here largely in PDF jargon, are by design not solely applicable PDF. You could georegister a portion DVI, AI, (E)PS or HTML page among others, or any file that references some Cartesian coordinate system like DGN or DWG. GeoTIFF has a provision for a CTM that maps pixels to a projected coordinate system. Lovely spec, that. I strongly encourage everyone to read it and profit from the work of Niles Ritter, Mike Ruth, and the GeoTIFF crew. I guess what I'm getting at is if there is a map or image in bits and bytes format, it should be georegistered! Perhaps this post should have been titled The Map Frame Manifesto...

A final note about map frames is that they are only implicitly tied to the content in or image on the page (we'll that's not exactly true with a variation of Adobe's georegistration technique, but I'll leave that for another post...). A page can have several map frames and they can "stack". For instance, maps in that road atlas under the Hanna Montana coloring book and crayons in the back of the car has a map of Utah in it, with insets for Salt Lake City and Provo obscuring Colorado and Nevada. In GeoPDF, that map would have three map frames: a base map for Utah and adjacent states with two insets on top for SLC and Provo, respectively. Map frame stacking is an important implementation detail, because it controls how you determine what you're looking at when, say doing roll-over detection for mouse movement. More on this, later.

So, georegistration is what makes GeoPDF GeoPDF?

Well... maybe yes, probably no. Or vice versa. It's become confusing. When TerraGo products create a GeoPDF file, they do more than just take data, make a picture of it, and stuff a little georegistration payload in the result. For instance, as Adam has pointed out, we take advantage of PDF features to provide a richer, more interactive, PDF file. However, many features often associated with GeoPDF are not unique to GeoPDF. Layers are not unique to GeoPDF. Object data are not unique to GeoPDF. These are "just" features of PDF that anyone with the time and patience can implement. ESRI, Bentley, Safe, and Cadcorp have all done it, although I'm not certain they all embed georegistration payloads.

In my mind, it's not GeoPDF without georegistration, that's for sure! As we work through the history and some of the implementation details in the history of GeoPDF, maybe we can sort this out to everyone's satisfaction.

Just where are Adobe's geospatial extensions to ISO 32000?

Reader GCP Maps provides a pointer to the where to actually see the Adobe extensions to ISO 32000, which I'll repeat here: Section 8.8.1 Geospatial Features in Adobe Supplement to the ISO 32000 BaseVersion: 1.7 ExtensionLevel: 3 [PDF]. That's page 49 if you want to jump right to it...

The GeoPDF CTM and Units: An Implementation Note

Like many projects, GeoPDF escaped from the lab before it was fully developed. An implementation detail that may seem bizarre at first glance are the units of the parameters CTM which defines the a GeoPDF georegistration. Recalling the CTM:

[ a b c d H V ]

the parameters a, b, c, and d are in units of meters per point. Parameters H and V are in meters. A point in PDF is defined as 1/72 of an inch. What's up with that?

If you take it all the way back, it goes back to the arcana of printing. We'll take it back as far as PostScript, where a unit in what's called default user space was defined to be a point, and a point defined as exactly 1/72 of an inch. See the Red Book [PDF] for more. The point unit default user space lives on in PDF, and, in GeoPDF.

Many of the interfaces for dealing with PDF pages take and yield values in points. The GEOTRANS projection engine we used at the time works in meters. The points-in-meters out (and vice versa) interface was convenient for bridging the two worlds and helped to make the georegistration concept -- this part of the page is "registered" to this part of the world -- concrete.

There's even a reason it's meters per point rather than the other way around. The eastings and northings associated with projections used for many practical mapping applications tend to be "large", often ranging around the hundreds of thousands or millions of meters. Points tend to range from zero to the low thousands for most practical page sizes. A ratio of points per meter often yielded fractional values that could not be encoded as a in a PDF file as a PDF number without significant loss of precision. Also, the H and V often took on values that could be recognized as being associated with this or that coordinate reference system. We ran into other issues with limitations of PDF numbers, the solution to which we'll leave for another time.

GeoPDF and Georegistration

One of the key concepts behind GeoPDF is georegistration. We define georegistration as a specific form of georeferencing in which a projected Cartesian coordinate system is registered to a page Cartesian coordinate system using a linear transformation. That is, the relationship between a point in a map to a point in a projected coordinate system is given by:

x = a h + c v + H
y = b h + d v + V

where (x, y) is the projected coordinate, (h, v) is the page coordinate and a, b, c, d, H, and V are the parameters that define the linear relationship. In matrix notation, this can be represented as:

|a c H||h|   |x|
|b d V||v| = |y|
|0 0 1||1|   |1|

We call the matrix a coordinate transformation matrix, or CTM. We encode this in a more compact form in PDF object notation as:

[ a b c d H V ]

where the square brackets delimit an array. More on PDF object notation later, but Adobe's PDF Architect and Senior Principal Scientist Jim King's tutorials give a nice overview.

Who does geospatial PDF?

I know that Adobe, ESRI, Safe, and Cadcorp all roll their own geospatial PDF files using Adobe's geospatial PDF extensions. Let me know if you or some one you know does as well.

Adobe groks GeoPDF.

Today GeoPDF files are created from software from TerraGo (naturally!), Adobe, ESRI,Cadcorp, ERDAS, Intergraph, and BAE Systems.

GeoPDF files are consumed by software from TerraGo (naturally!), Adobe, and ESRI.

Since we support Adobe's geospatial PDF extensions and are going to use them under the hood moving forward, the distinction between the georegistration techniques from an us-v-them perspective are, more or less, moot.

How do I get my hands on spec?

If you would like to create a geospatial PDF file, I strongly recommend georegistration using Adobe's extensions to ISO 32000. Apparently, OGC 08-139r1 has not been posted yet to the OGC best practices repository. I'm lobbying to post it on our corporate website... we'll see how that goes. If you want to get your hands on the spec to get cracking making an indexer or something like that, let me know and I'll get you a copy and some implementation notes. If you want to just see how we did it, open the PDF in a text editor and search for LGIDict... I would like to start dissecting what we did and why on this blog in a series of posts.

Closing out Bug 1584 "Make GeoPDF Open Source"

Customer IANT logged the following enhancement request:

I know it's not going to happen but had a request to make GeoPDF open source and allow how we store all of the georegistration information publicly available.

IANT, it happened. Bug 1584 is closed, resolved "fixed". Cheers!

What's Your Beef with Calling OGC 08-139r1 GeoPDF 2.2?

I've been asked by different folks why I think that conflating GeoPDF with the details of a georegistation technique is not a good idea. My main beef is that it confines GeoPDF to some implementation details and sets it against/in contrast to whatever is georegistered using Adobe's proposed extensions to ISO 32000. I always had a more inclusive vision for GeoPDF. There are several PDF standards: PDF/A, PDF/E, PDF/X, PDF/UA... I imagine that there might be a PDF/G some day (or, perhaps, extensions to PDF/E, e.g., PDF/EG), and I would like to say that "this is a PDF/G-compliant GeoPDF file!" I would like this blog to be a natural home for all things geospatial PDF.

However, this is merely a matter of opinion and taste. What more concerns me is that it creates the false illusion that there is some sort of specification competition where there is none. Adobe has promulgated extensions to ISO 32000 and that's the path toward the future of PDF/G (or whatever) standard is coming down the pike to be considered and approved by ISO. We support this in our products and our planning.

What's delightful is that there is a published and unencumbered specification of a technique for georegistering PDF files -- finally! This is something that I've fighting for and been working to see since 2003. I'm looking forward to Google and Yahoo! bringing me beautiful maps and imagery with queries like "battle of atlanta" or "landsat new zealand 1999-2009" and being able to actually look at it because it's in the PDF and play with it because it's georegistered. By publishing our old technique, search engines will be able to index GeoPDF files out there on the web regardless of how they were georegistered.

OGC Approves GeoPDF 2.2 as OGC Best Practice

GeoPDF 2.2 OGC 08-139r1 was approved as OGC Best Practice. Personally, I think that the use of GeoPDF was unfortunate, for various reasons, but I'm happy to see the georegistration technique published. If you'd like to roll-your-own geospatial PDF, I'd recommend using Adobe's proposed geospatial extensions to ISO 32000. Our software supports this georegistration technique, and we're already moving in that direction.

Placemark Balloon Enhancement in GE 5.0

Many of you may have noticed that the KML generated from your application does not properly reference any placemark balloon attachments when viewed in Google Earth. This is because those attachments are referenced with a local file path rather than a valid URL that Google Earth is looking for. There used to be a gross work around to get those attached files to display properly which involved web servers and modifications to the KML itself. No More! Google Earth 5.0 added a provision to accept local file paths so now all referenced files in the KML will work regardless of whether or not they are on a web server!

To enable this functionality, download Google Earth 5.0 and then go to Tools --> Options --> General tab and check the box that says “Allow placemark balloons to access local files and personal data.”

I understand that Google Earth is more of a wiz-bang visualization utility than an actual geospatial analysis tool but darn it, people use it! This small modification now offers the KML from 3rd party applications whos users don't always have the ability to place their data web server a chance for tighter integration...

Google Earth 5.0 is released!

Google announced today that Google Earth 5.0 has officially been released. This latest release appears to have a few interesting new features that will allow you to view historical imagery from around the globe, ocean floor and surface data as well as the ability to record your GE session!  Check out the online tour by clicking here.  Enjoy!

UPDATE: More information can be found on Google's official blog site...