Answer by Frank Heikens for New PostgreSQL 9.1 configuration doesn’t work
pg_config_manual.h is not a configuration file, it's part of the source code. After a change in the source code, you have to compile all the code and install the program you just compiled.Changing the...
View ArticleAnswer by Frank Heikens for Ways to speed up IN queries under PostgreSQL
IN() using many parameters will result in many cases in a sequential table scan. That might be slow, depending on table size and speed of your system.Create a temporary table with all your variables...
View ArticleAnswer by Frank Heikens for Huge database logging of event type rows and ways...
We have something simular, ~5000 events per seconds. Tables are partitioned by month and we store 5 years of data. That gives us a 60 partitions, works fine. It runs on PostgreSQL version 9.1, works...
View ArticleAnswer by Frank Heikens for Created user can access all databases in...
PUBLIC has access to the database by default, but it can't access the data. You can REVOKE the PUBLIC:REVOKE CONNECT ON DATABASE your_database FROM PUBLIC;If you want this setting for all future...
View ArticleAnswer by Frank Heikens for Perf of query of large table by date?
We do 500 to 3000 inserts per second on a table, but in partitions of 1 month, and are monitoring the last 2 hours of activity. An index on the timestamp does the trick, most queries run less than a...
View ArticleAnswer by Frank Heikens for Automating failover in PostgreSQL 9.1
Check out repmrg:repmgr is a set of open source tools that helps DBAs and System administrators manage a cluster of PostgreSQL databases..By taking advantage of the Hot Standby capability introduced in...
View ArticleAnswer by Frank Heikens for Postgresql: Two concurrent COPY FROM
It just works, like it should. It doesn't lock the table, so start testing.
View ArticleAnswer by Frank Heikens for Trying to create PostGIS database
You must install the PostGIS functions, data types, etc. before you can use PostGIS. This can be found in these three SQL-files. Set the correct path to the files and execute them in PostgreSQL by...
View ArticleAnswer by Frank Heikens for Unit testing of stored procedures
For PostgreSQL, check out pgTAP:pgTAP is a suite of database functions that make it easy to write TAP-emitting unit tests in psql scripts or xUnit-style test functions.
View ArticleAnswer by Frank Heikens for Streaming replication and failover on PostgreSQL
In our shop we selected repmgr and pgbouncer instead of pgpool. repmgr has some nice tooling to setup and maintain the cluster of replicated database servers. In our case 1 master and 2 slaves (one...
View ArticleAnswer by Frank Heikens for Can you make SSL and SSL-only (hostssl)...
I'm not sure about pgPool, but pgBouncer (another connection pool) can use SSL by using Stunnel.Check your functional requirements to see if pgBouncer can solve your problem. Maybe pgPool can also use...
View ArticleAnswer by Frank Heikens for Benefits of using backtick (`) in MySQL queries?
If you want to use something around object identifiers, use at least the standard double quotes: "This works in MySQL, PostgreSQL, SQL Server, Oracle, etc. etc. For MySQL you might need the SQL...
View ArticleAnswer by Frank Heikens for SQL Server Express vs. Oracle Express Edition vs...
In my view, DB2 is the clear cut winner.What about PostgreSQL:cores: unlimitedmem: unlimitedsize: unlimitedlicense: PostgreSQL (BSD style)imho, PostgreSQL beats all Express/cripled databases.
View ArticleAnswer by Frank Heikens for What GUI tools can be used for PostgreSQL in Mac...
Check the wiki page, many different tools available.
View ArticleAnswer by Frank Heikens for Too many TIME_WAIT connections in PostgreSQL
1500 to 2200 concurrent connections are only useful when you need them. Otherwise, bring the total number of open connections down and start using a connection pool like pgBouncer or PgPool-II. See...
View ArticleAnswer by Frank Heikens for PostgreSQL replication and large import
PostgreSQL out of the box replication has one master and can have multiple (cascading, as of version 9.2) slaves. If you need multi master replication, 2 masters in your case, you need Postgres-XC.My...
View ArticleAnswer by Frank Heikens for Locking issue with concurrent DELETE / INSERT in...
Use a DEFERRABLE primary key and try again.
View ArticleAnswer by Frank Heikens for Manually increase PostgreSQL's table size limit...
Just use a different setting for the block size:--with-blocksize=BLOCKSIZEThe default, 8 kilobytes, is suitable for most situations; but other values may be useful in special cases. The value must be a...
View ArticleAnswer by Frank Heikens for Does running pg_dump on live db produce...
From the manual:It makes consistent backups even if the database is being used concurrently.So yes, you can trust the backup. Of course, it's PostgreSQL, you can trust your data in PostgreSQL.
View ArticleAnswer by Frank Heikens for PostgreSQL procedural languages overhead...
Is the context a big overhead? Can I use it for realtime data mapping (let's say 1000 queries/s))Performance depends on hardware and complexity of your functions. I created an appliance that ran on a...
View ArticleAnswer by Frank Heikens for PostgreSQL partitioning query performance...
What version do you use? Version 9.1 has a major improvement for partitioning, check this blog from Greg Smith.When using version 9.1, I get this result for this queryEXPLAIN ANALYZESELECT MAX(logdate)...
View ArticleAnswer by Frank Heikens for How do I list all databases and tables using psql?
Please note the following commands:\list or \l: list all databases\c <db name>: connect to a certain database\dt: list all tables in the current database using your search_path\dt *.: list all...
View ArticleAnswer by Frank Heikens for Best way to implement non-mergeable view in...
I would start using JOIN's as the new standard, since 30 years ago, ANSI SQL-92.And when you need data from t1 and t2, there is no need for joining t3: Use EXISTS to check for the condition.To improve...
View ArticleAnswer by Frank Heikens for Why is my mean much more bigger than my Execution...
I get the impression that Hyperfine measures the complete end-to-end call, including getting access to the container, starting psql and executing the query. If you want to measure just the query,...
View ArticleAnswer by Frank Heikens for Can a new transaction claim an older sequence id?
A sequence is a simple number generator. When you ask for a number, it gives you a new one. Unless you mess with the settings of the sequence or use a wraparound, you will never ever get a number that...
View ArticleAnswer by Frank Heikens for PostgreSQL alternative to SQL Server’s `try_cast`...
It's a fairly old question but the answer is now sort of obsolete. Since PostgreSQL version 16 you can use the function pg_input_is_valid() to see if your input is valid for your data type:SELECT...
View ArticleAnswer by Frank Heikens for Why PostgreSQL v10 and v16 have slightly...
The behavior of pg_dump changed with version 14:Allow pg_dump to dump only certain extensionsOlder versions dump all extensions in the backup, since version 14 by default all non-system extensions in...
View ArticleAnswer by Frank Heikens for How to set data_directory as /data and pg_wal...
Set both --pgdata= and --waldir= when initdb creates a new PostgreSQL database cluster.
View ArticleComment by Frank Heikens on IndexOptimize on VLDBs with tight maintenance window
Why do you even do it? What problem should it solve? And does it make your queries faster?
View ArticleComment by Frank Heikens on No config, data files and cluster after...
Check askubuntu.com
View ArticleComment by Frank Heikens on How to use an alias of a computed column from...
and projects.status in ('started', 'closed', 'rejected', 'canceled') is a strange combination with and closed > '2025-02-01'. Only records with the status closed can match, none of the other since...
View ArticleComment by Frank Heikens on does postgresql support unsigned bigint data type
When 9223372036854775807 IDs are insufficient, you most likely need a UUID. How many records per second do you create?
View ArticleComment by Frank Heikens on Why all Postgresql workers are waiting for a...
shared_buffers = 64GB is too much for a 128GB machine. It should be 32GB max. Other settings are also unusual.
View ArticleComment by Frank Heikens on Why all Postgresql workers are waiting for a...
@Leon The database configuration is pretty bad; you should change that first. Such a small database with so much RAM will live in memory, just like a memory database. A Redis database with the wrong...
View ArticleComment by Frank Heikens on Best Database for Bulk Reads and Fast Key Lookups...
might outperform is always true. And if not today, then tomorrow or next month. Besides, most performance limitations are sitting behind the keyboard and drinking coffee or tea. A dbms does much more...
View ArticleComment by Frank Heikens on Best way to achieve HIPAA compliance in a...
You can't be HIPAA compliant just because you did something (right) in the database. The database is just part of it.
View ArticleComment by Frank Heikens on Do I need to CREATE LANGUAGE plpgsql in...
Currently, only versions 13+ are supported by the PostgreSQL project. When your code depends on plpgsql, you should at least check for plpgsql: SELECT EXISTS(SELECT FROM pg_extension WHERE extname =...
View ArticleComment by Frank Heikens on Pattern match with CTE
Why do you want to use a CTE for a WHERE condition? And why do you add the braces {} to the ids? If you want to return a JSON object, use the JSON functions to create JSON.
View ArticleAnswer by Frank Heikens for PostgreSQL query stuck on LWLock:BufferMapping...
maintenance_work_mem is very low for a 104 GB database server that has 15TB storage. I would change this one and give it at least a few GB of RAM.But also check the rest of your configuration and...
View ArticleComment by Frank Heikens on Poor performance bucketing historical data with...
Please share the results of your SQL statement from explain(analyze, verbose, buffers, settings). All in plain text as an update of your question.
View ArticleAnswer by Frank Heikens for Poor performance bucketing historical data with...
I think your current query is wrong. Please check this one:SELECT CAST(day AS date) AS day , COALESCE(SUM(bad),0) AS bad , COALESCE(SUM(awful),0) AS awful , COALESCE(SUM(catastrophic),0) AS...
View Article