I did some benchmarking and it turns out there is real no measurable
difference between putting the data in there and not doing it. So
we'll just always do it. This means that it doesn't depend on the
-x/--extra-attributes command line option any more.
Note that the attributes will still be nil when coming from the
middle in update mode if -x/--extra-attributes was not used on import.
Add new get_bbox() function for geometries in Lua. Like the get_bbox()
function on OSM objects it returns the bounding box as four numbers in a
Lua array (table). Having this function on geometries allows for more
flexibility, you can call it on a centroid or a projected geometry for
instance.
It is faster to call get_bbox() on the OSM object, because it removes
the step of building the geometry first, so for those cases where this
is good enough the old use is fine and is still supported.
See #2219
COPYs are handled in extra threads through a worker queue. Using
std::variant instead of virtual classes simplifies the code, because we
don't need the indirection through a unique_ptr any more.
No flex config file, or one that doesn't define any tables is usually
not very useful. But it can be used in slim mode to have the middle
tables only. Or it can be used to do some other processing or instead
if the null output for testing.
This change allows an empty config file, but a warning is printed that
no output tables are defined. You still have to specify a config file,
it can be an empty one, but it has to be there. This is so that if the
user forgets naming a config file on the command line they get an error
and not only a warning that might be missed.
We are moving to the flex output which always needs Lua. Making Lua
optional isn't useful any more and just makes code more complicated.
This commit makes Lua non-optional.
New nodes or ways (ie with version=1) can't have parent ways or
relations. So we don't have to add them to the list of "changed nodes"
or "changed ways" to trigger re-processing of changed parents. This will
save us a whole lot of effort looking for parents that are never there.
Note that we are checking for version != 1. In the (unlikely) case that
there are no versions in the input file (making version == 0),
everything will work as before.
Remove has_pending() which checks two things we can easily check each by
itself and sometimes only need one or the other.
Turn get_pending_way_ids() and get_pending_relation_ids() into simple
getters, they are only needed for testing outside the class.
Now that we always use dependency management, we don't need the
dependency manager to be a virtual class any more. In fact it isn't
really doing that much, so we can roll it into the code in osmdata_t.
Adds a "context" parameter to database connection setup describing what
this connection is for. This is something like "middle.query" or
"out.pgsql". It will be added to the application name, so querying
pg_stat_activity will show it like "osm2pgsql.middle.query/C2" etc.
The connection parameters (such as dbname, user, host, ...) are stored
in their original format and not concatenated into a conninfo string any
more. We are now using a different database connection function from
libpq that takes them in this form. This allows us to get rid of the
code that transforms those parameters into the conninfo string that
takes various forms of connection parameters into account, especially
around the URI form of the connection parameters. The libpq has this
code anyway, see also the last parameter "expand_dbname" of the
PQconnectdbParams() function
(https://www.postgresql.org/docs/current/libpq-connect.html).
This also allow us to have slightly different database connection
parameters for different database connection. This is used here to add
the connection id into the fallback_application_name parameter. This
connection id also reported when using --log-sql. This way you can
correlate the ids from the log with the information shown in
pg_stat_activity.
This commit looks large, because it changes the type std::string to
connection_parameters_t in many places. I have kept the parameter name
"conninfo" in an effort to keep the commit small, the next commit will
change that also.
This switches parsing of command line arguments from getopt to the CLI11
library.
Its adds CLI11 as a new dependency. With this commit CLI11 is vendored
in in the contrib directory, because it is not available in Ubuntu 20.04
and Debian before Bookworm. It is available in Ubuntu 22.04 und Debian
Bookworm and it is available in Homebrew and as vcpkg. So a bit down the
line we can switch to the versions available from the OS distribution.
For Windows we don't need the getopt version from alex85k any more and
this commit also removes that code from the Github action setup.
This commit is quite large, because we have to change everything at the
same time. Changes are for osm2pgsql and osm2pgsql-gen which now share
part of the option parsing code (database and logging options), in the
new src/command-line-app.[ch]pp files.
The code tries to keep the functionality the same as much as possible.
So no extra checks on command line options or so, these can come later.
There are some unavoidable changes:
* Some error messages have changed
* Help output is now generated by the CLI11 library so it looks very
different. There is no verbose help version any more. All command
line options are shown with --help, but some with a bit less detail
than before. But all the detail is in the man page anyway.
* CLI11 can not parse an empty option parameter, so something like
"--foo=" does not work, it tries to use the next option as value
for the option. This is not very relevant in the real world, but
there might be some corner cases where this changes behaviour.
See #142
Jochen Topf
Sarah Hoffmann