Answer by Frank Heikens for New PostgreSQL 9.1 configuration doesn’t work
pg_config_manual.h is not a configuration file, it's part of the source code. After a change in the source code, you have to compile all the code and install the program you just compiled.Changing the...
View ArticleAnswer by Frank Heikens for Ways to speed up IN queries under PostgreSQL
IN() using many parameters will result in many cases in a sequential table scan. That might be slow, depending on table size and speed of your system.Create a temporary table with all your variables...
View ArticleAnswer by Frank Heikens for Huge database logging of event type rows and ways...
We have something simular, ~5000 events per seconds. Tables are partitioned by month and we store 5 years of data. That gives us a 60 partitions, works fine. It runs on PostgreSQL version 9.1, works...
View ArticleAnswer by Frank Heikens for Created user can access all databases in...
PUBLIC has access to the database by default, but it can't access the data. You can REVOKE the PUBLIC:REVOKE CONNECT ON DATABASE your_database FROM PUBLIC;If you want this setting for all future...
View ArticleAnswer by Frank Heikens for Perf of query of large table by date?
We do 500 to 3000 inserts per second on a table, but in partitions of 1 month, and are monitoring the last 2 hours of activity. An index on the timestamp does the trick, most queries run less than a...
View ArticleAnswer by Frank Heikens for Automating failover in PostgreSQL 9.1
Check out repmrg:repmgr is a set of open source tools that helps DBAs and System administrators manage a cluster of PostgreSQL databases..By taking advantage of the Hot Standby capability introduced in...
View ArticleAnswer by Frank Heikens for Postgresql: Two concurrent COPY FROM
It just works, like it should. It doesn't lock the table, so start testing.
View ArticleAnswer by Frank Heikens for Trying to create PostGIS database
You must install the PostGIS functions, data types, etc. before you can use PostGIS. This can be found in these three SQL-files. Set the correct path to the files and execute them in PostgreSQL by...
View ArticleAnswer by Frank Heikens for Unit testing of stored procedures
For PostgreSQL, check out pgTAP:pgTAP is a suite of database functions that make it easy to write TAP-emitting unit tests in psql scripts or xUnit-style test functions.
View ArticleAnswer by Frank Heikens for Streaming replication and failover on PostgreSQL
In our shop we selected repmgr and pgbouncer instead of pgpool. repmgr has some nice tooling to setup and maintain the cluster of replicated database servers. In our case 1 master and 2 slaves (one...
View ArticleAnswer by Frank Heikens for Can you make SSL and SSL-only (hostssl)...
I'm not sure about pgPool, but pgBouncer (another connection pool) can use SSL by using Stunnel.Check your functional requirements to see if pgBouncer can solve your problem. Maybe pgPool can also use...
View ArticleAnswer by Frank Heikens for Benefits of using backtick (`) in MySQL queries?
If you want to use something around object identifiers, use at least the standard double quotes: "This works in MySQL, PostgreSQL, SQL Server, Oracle, etc. etc. For MySQL you might need the SQL...
View ArticleAnswer by Frank Heikens for SQL Server Express vs. Oracle Express Edition vs...
In my view, DB2 is the clear cut winner.What about PostgreSQL:cores: unlimitedmem: unlimitedsize: unlimitedlicense: PostgreSQL (BSD style)imho, PostgreSQL beats all Express/cripled databases.
View ArticleAnswer by Frank Heikens for What GUI tools can be used for PostgreSQL in Mac...
Check the wiki page, many different tools available.
View ArticleAnswer by Frank Heikens for Too many TIME_WAIT connections in PostgreSQL
1500 to 2200 concurrent connections are only useful when you need them. Otherwise, bring the total number of open connections down and start using a connection pool like pgBouncer or PgPool-II. See...
View ArticleAnswer by Frank Heikens for PostgreSQL replication and large import
PostgreSQL out of the box replication has one master and can have multiple (cascading, as of version 9.2) slaves. If you need multi master replication, 2 masters in your case, you need Postgres-XC.My...
View ArticleAnswer by Frank Heikens for Locking issue with concurrent DELETE / INSERT in...
Use a DEFERRABLE primary key and try again.
View ArticleAnswer by Frank Heikens for Manually increase PostgreSQL's table size limit...
Just use a different setting for the block size:--with-blocksize=BLOCKSIZEThe default, 8 kilobytes, is suitable for most situations; but other values may be useful in special cases. The value must be a...
View ArticleAnswer by Frank Heikens for Does running pg_dump on live db produce...
From the manual:It makes consistent backups even if the database is being used concurrently.So yes, you can trust the backup. Of course, it's PostgreSQL, you can trust your data in PostgreSQL.
View ArticleAnswer by Frank Heikens for PostgreSQL procedural languages overhead...
Is the context a big overhead? Can I use it for realtime data mapping (let's say 1000 queries/s))Performance depends on hardware and complexity of your functions. I created an appliance that ran on a...
View ArticleAnswer by Frank Heikens for PostgreSQL partitioning query performance...
What version do you use? Version 9.1 has a major improvement for partitioning, check this blog from Greg Smith.When using version 9.1, I get this result for this queryEXPLAIN ANALYZESELECT MAX(logdate)...
View ArticleAnswer by Frank Heikens for How do I list all databases and tables using psql?
Please note the following commands:\list or \l: list all databases\c <db name>: connect to a certain database\dt: list all tables in the current database using your search_path\dt *.: list all...
View ArticleAnswer by Frank Heikens for Best way to implement non-mergeable view in...
I would start using JOIN's as the new standard, since 30 years ago, ANSI SQL-92.And when you need data from t1 and t2, there is no need for joining t3: Use EXISTS to check for the condition.To improve...
View ArticleAnswer by Frank Heikens for Why is my mean much more bigger than my Execution...
I get the impression that Hyperfine measures the complete end-to-end call, including getting access to the container, starting psql and executing the query. If you want to measure just the query,...
View ArticleAnswer by Frank Heikens for Can a new transaction claim an older sequence id?
A sequence is a simple number generator. When you ask for a number, it gives you a new one. Unless you mess with the settings of the sequence or use a wraparound, you will never ever get a number that...
View ArticleAnswer by Frank Heikens for PostgreSQL alternative to SQL Server’s `try_cast`...
It's a fairly old question but the answer is now sort of obsolete. Since PostgreSQL version 16 you can use the function pg_input_is_valid() to see if your input is valid for your data type:SELECT...
View ArticleAnswer by Frank Heikens for Why PostgreSQL v10 and v16 have slightly...
The behavior of pg_dump changed with version 14:Allow pg_dump to dump only certain extensionsOlder versions dump all extensions in the backup, since version 14 by default all non-system extensions in...
View ArticleAnswer by Frank Heikens for How to set data_directory as /data and pg_wal...
Set both --pgdata= and --waldir= when initdb creates a new PostgreSQL database cluster.
View ArticleAnswer by Frank Heikens for PostgreSQL query stuck on LWLock:BufferMapping...
maintenance_work_mem is very low for a 104 GB database server that has 15TB storage. I would change this one and give it at least a few GB of RAM.But also check the rest of your configuration and...
View ArticleComment by Frank Heikens on Is pg_dump as acceptable as pg_dumpall for...
I'm troubleshooting an error on the new system We'd love to help you, but we do need at least the error message. pg_dumpall also dumps the global objects that are not part of a single database, like...
View ArticleComment by Frank Heikens on Error "could not reserve shared memory region ......
It’s a misconfigured system, for Windows you should use very low settings for shared buffers.
View ArticleComment by Frank Heikens on PostgreSQL 13 on Windows –“invalid page in block...
Try to make a backup with just the data: pg_dump --data-only -d your_database
View ArticleComment by Frank Heikens on is this a good pratice, using varchar as data...
Yes, see Kimball: kimballgroup.com/data-warehouse-business-intelligence-resources/…
View ArticleComment by Frank Heikens on Copy pasting timestampz values don't work on...
@anta40 Why do you cast your column to a varchar? If this is a UUID, you just destroyed the option to use the index for this column to find the correct record
View ArticleComment by Frank Heikens on What will be the root cause for the incident?
Please share the query, the DDL for all tables and indexes involved, and the results from explain(analyze, verbose, buffers, settings) for this query. All in plain text, as an update to the question....
View ArticleComment by Frank Heikens on Is it bad practice to just attempt to delete rows...
But I can also just simply occasionally try to delete the rows and skip the ones where this fails because of a foreign key constraint violation. Sounds like a plan! It's simple, reliable, and...
View ArticleComment by Frank Heikens on How can different types of indexes in PostgreSQL...
You didn’t share any information which makes it impossible to help you. Please check the manual about how to get information from your database: postgresql.org/docs/current/performance-tips.html
View ArticleComment by Frank Heikens on Truncate partitions with foreign key to a...
Your foreign key references the parent table, not the table partition directly. What you want is, to my knowledge, not supported by pg_partman.
View ArticleComment by Frank Heikens on Postgresql XID wrap around prevention overloading...
You could set vacuum_freeze_min_age to 0 for these partitions. That will freeze all tuples. See also stackoverflow.com/questions/70179712/…
View ArticleComment by Frank Heikens on Postgres recovery
Could you share some information about what PostgreSQL version, what OS, any error messages in the logfiles, etc.
View ArticleComment by Frank Heikens on Postgresql XID wrap around prevention overloading...
@zerz If you can't drop the old (empty?) partitions, you might be able to merge them. Instead of 100k empty tables, just one or two.
View ArticleAnswer by Frank Heikens for Truncate partitions with foreign key to a...
The first issue is what TRUNCATE can and can not do. Or actually WILL NOT DO. See this part of the manual:TRUNCATE cannot be used on a table that has foreign-key referencesfrom other tables, unless all...
View Article