Comment by Frank Heikens on Continious aggregates (keeping aggregated data in...
Why all the complexity for just a couple of million records? To be honest, I get the impression that the complexity is what makes things slow. Start with some good enough SQL statements and you should...
View ArticleComment by Frank Heikens on PostgreSQL date index
This query doesn't search for anything, it takes all records including everything. Counting all records in a table, isn't the best part of PostgreSQL and we avoid this (for large tables) by maintaining...
View ArticleComment by Frank Heikens on Optimize query with many joins
The current query gets bogged down on tab5 (wonderful name, someone should be fired for this). Without any background of what the query should do, what the data model is and some data, I can only...
View ArticleComment by Frank Heikens on Move row of a table to an inherited table in...
Why do you use inheritance in the first place? It exists, but it also has its limitations and quirks.
View ArticleComment by Frank Heikens on Should we re-run pg_prewarm('table') after data...
Does it solve a problem for you? If so, what is the problem you're facing?
View ArticleComment by Frank Heikens on Update row data with dynamic lag
That's not how a database works since records don't have an order without an order by in your SELECT statement. There is no relation between records if you don't create that relation. It's the...
View ArticleComment by Frank Heikens on Sensible value for autovacuum_cost_delay
The VACUUM part doesn't make sense to me: TRUNCATE rebuilds the table and doesn't need VACUUM and inserting new data won't trigger a VACUUM either, only ANALYZE. What is missing in this picture?
View ArticleComment by Frank Heikens on Optimizing type alignment for table with many of...
Partitioning on the ts-column can make the delete really cheap: Just drop or truncate the partition.
View ArticleComment by Frank Heikens on Optimized count for large table using triggers,...
Without a query, the DDL for all tables and indexes involved and the result from explain(analyze, verbose, buffers, settings) it's impossible to improve the performance of this query. Could you please...
View ArticleComment by Frank Heikens on Why is restored database faster than original?
Did you try VACUUM FULL on the original database? Rewriting all tables to optimize storage. When you use explain, please use explain(analyze, verbose, buffers, settings) to get the whole picture,...
View ArticleComment by Frank Heikens on Why does a query with extra left join work faster?
Could you please share the results from explain (analyze, verbose, buffers, settings) for your SQL statements? You only showed the plan, not the execution of the plan
View ArticleComment by Frank Heikens on Optimize query performance for a filter on col_a...
Could you please share the result from explain(analyze, verbose, buffers, settings) for your SQL statement, the statement itself and the DDL for all tables and indexes involved? All in plain text, as...
View ArticleAnswer by Frank Heikens for How do I specify that a column should be...
two options: Use the "datatype"SERIAL or create a sequence and use this sequence as a default value for your integer:CREATE SEQUENCE your_seq;CREATE TABLE foo( id int default...
View ArticleAnswer by Frank Heikens for Are there any good and free tools for managing a...
pgAdmin and psql, those are the tools I use. A PHP-script isn't very usefull for database management, too slow and many problems with transactions: What to do when something goes wrong? Commit or...
View ArticleAnswer by Frank Heikens for How much could sequential scan gain by GPU?
You'd better ask this question on the hackers mailing list, there has been some debate about this subject in the past. The PostgreSQL developers can tell you much more about...
View ArticleAnswer by Frank Heikens for Visualization of MySQL/PostgreSQL table structure...
You mean something like SQL Power Architect? The open source version can be found here.
View ArticleAnswer by Frank Heikens for Small sample database for practicing database...
Take a look at the Dell DVD Store database. It's available for multiple databases, SQL Server included.The Dell DVD Store is an open source simulation of an online ecommerce site with implementations...
View ArticleAnswer by Frank Heikens for PostgreSQL: reuse of complex intermediate result...
You need an index on test1_id in table test2, that will change things.Seq Scan on test1 t1 (cost=0.00..301450.63 rows=8761 width=12) (actual time=0.108..229.859 rows=8761 loops=1) SubPlan 1 ->...
View ArticleAnswer by Frank Heikens for New PostgreSQL 9.1 configuration doesn’t work
pg_config_manual.h is not a configuration file, it's part of the source code. After a change in the source code, you have to compile all the code and install the program you just compiled.Changing the...
View ArticleAnswer by Frank Heikens for Ways to speed up IN queries under PostgreSQL
IN() using many parameters will result in many cases in a sequential table scan. That might be slow, depending on table size and speed of your system.Create a temporary table with all your variables...
View ArticleAnswer by Frank Heikens for Huge database logging of event type rows and ways...
We have something simular, ~5000 events per seconds. Tables are partitioned by month and we store 5 years of data. That gives us a 60 partitions, works fine. It runs on PostgreSQL version 9.1, works...
View ArticleAnswer by Frank Heikens for Created user can access all databases in...
PUBLIC has access to the database by default, but it can't access the data. You can REVOKE the PUBLIC:REVOKE CONNECT ON DATABASE your_database FROM PUBLIC;If you want this setting for all future...
View ArticleAnswer by Frank Heikens for Perf of query of large table by date?
We do 500 to 3000 inserts per second on a table, but in partitions of 1 month, and are monitoring the last 2 hours of activity. An index on the timestamp does the trick, most queries run less than a...
View ArticleAnswer by Frank Heikens for Automating failover in PostgreSQL 9.1
Check out repmrg:repmgr is a set of open source tools that helps DBAs and System administrators manage a cluster of PostgreSQL databases..By taking advantage of the Hot Standby capability introduced in...
View ArticleAnswer by Frank Heikens for Postgresql: Two concurrent COPY FROM
It just works, like it should. It doesn't lock the table, so start testing.
View ArticleAnswer by Frank Heikens for Trying to create PostGIS database
You must install the PostGIS functions, data types, etc. before you can use PostGIS. This can be found in these three SQL-files. Set the correct path to the files and execute them in PostgreSQL by...
View ArticleAnswer by Frank Heikens for Unit testing of stored procedures
For PostgreSQL, check out pgTAP:pgTAP is a suite of database functions that make it easy to write TAP-emitting unit tests in psql scripts or xUnit-style test functions.
View ArticleAnswer by Frank Heikens for Streaming replication and failover on PostgreSQL
In our shop we selected repmgr and pgbouncer instead of pgpool. repmgr has some nice tooling to setup and maintain the cluster of replicated database servers. In our case 1 master and 2 slaves (one...
View ArticleAnswer by Frank Heikens for Can you make SSL and SSL-only (hostssl)...
I'm not sure about pgPool, but pgBouncer (another connection pool) can use SSL by using Stunnel.Check your functional requirements to see if pgBouncer can solve your problem. Maybe pgPool can also use...
View ArticleAnswer by Frank Heikens for Benefits of using backtick (`) in MySQL queries?
If you want to use something around object identifiers, use at least the standard double quotes: "This works in MySQL, PostgreSQL, SQL Server, Oracle, etc. etc. For MySQL you might need the SQL...
View ArticleAnswer by Frank Heikens for SQL Server Express vs. Oracle Express Edition vs...
In my view, DB2 is the clear cut winner.What about PostgreSQL:cores: unlimitedmem: unlimitedsize: unlimitedlicense: PostgreSQL (BSD style)imho, PostgreSQL beats all Express/cripled databases.
View ArticleAnswer by Frank Heikens for What GUI tools can be used for PostgreSQL in Mac...
Check the wiki page, many different tools available.
View ArticleAnswer by Frank Heikens for Too many TIME_WAIT connections in PostgreSQL
1500 to 2200 concurrent connections are only useful when you need them. Otherwise, bring the total number of open connections down and start using a connection pool like pgBouncer or PgPool-II. See...
View ArticleAnswer by Frank Heikens for PostgreSQL replication and large import
PostgreSQL out of the box replication has one master and can have multiple (cascading, as of version 9.2) slaves. If you need multi master replication, 2 masters in your case, you need Postgres-XC.My...
View ArticleAnswer by Frank Heikens for Locking issue with concurrent DELETE / INSERT in...
Use a DEFERRABLE primary key and try again.
View ArticleAnswer by Frank Heikens for Manually increase PostgreSQL's table size limit...
Just use a different setting for the block size:--with-blocksize=BLOCKSIZEThe default, 8 kilobytes, is suitable for most situations; but other values may be useful in special cases. The value must be a...
View ArticleAnswer by Frank Heikens for Does running pg_dump on live db produce...
From the manual:It makes consistent backups even if the database is being used concurrently.So yes, you can trust the backup. Of course, it's PostgreSQL, you can trust your data in PostgreSQL.
View ArticleAnswer by Frank Heikens for PostgreSQL procedural languages overhead...
Is the context a big overhead? Can I use it for realtime data mapping (let's say 1000 queries/s))Performance depends on hardware and complexity of your functions. I created an appliance that ran on a...
View ArticleAnswer by Frank Heikens for PostgreSQL partitioning query performance...
What version do you use? Version 9.1 has a major improvement for partitioning, check this blog from Greg Smith.When using version 9.1, I get this result for this queryEXPLAIN ANALYZESELECT MAX(logdate)...
View ArticleAnswer by Frank Heikens for How do I list all databases and tables using psql?
Please note the following commands:\list or \l: list all databases\c <db name>: connect to a certain database\dt: list all tables in the current database using your search_path\dt *.: list all...
View ArticleAnswer by Frank Heikens for Best way to implement non-mergeable view in...
I would start using JOIN's as the new standard, since 30 years ago, ANSI SQL-92.And when you need data from t1 and t2, there is no need for joining t3: Use EXISTS to check for the condition.To improve...
View Article
