F.24. lt_sql_inspect

F.24.1. SQL best practices
F.24.2. SQL interception

The lt_sql_inspect module provides two features: SQL best practices and SQL interception

F.24.1. SQL best practices

F.24.1.1. Overview

This feature will provides rules to standardize sql writing.

There are two parts to this: LightDB DDL Specification Check and LightDB DML Specification Check.

F.24.1.2. Configuration Parameters

lightdb_best_practice_mode (String)

lightdb_best_practice_mode is used to specify the rules to check sql.

Now it support following rules: keyword_not_allowed,bad_datatype,must_have_pk,must_have_gmt,no_more_100cols,no_paging. The default value is 'must_have_pk'.

This setting can be changed within individual sessions.

lightdb_best_practice_level (String)

lightdb_best_practice_level is used to specify the message level when check failed.

It is used for both SQL best practices and SQL interception.

This setting can be changed within individual sessions.

F.24.1.3. LightDB DDL Specification Check

When you use DDL statements, LightDB will first check whether your statements comply with the specification.

At present, there are five checks: keyword check (including table name, field name, library name and index name), field type check (float,double,text,real, uuid,bytea,boolean,bit,character), whether the table must contain primary key, whether the table contains fields of creation time and last modification time, and the number of columns for operation does not exceed 100.The DDL statement specification supports:

CREATE DATABASE...
CREATE TABLE...
CREATE INDEX...
ALTER TABLE...

The keyword list can be referenced "www.hs.net/lightdb/docs/current/sql-keywords-appendix.html"

When you use nonstandard statements, you will report warnings or errors directly. For example:

create database text;
WARNING:  LightDB DDL check warn! keyword not allowed with [db name]:text

create table text(id int);
WARNING:  LightDB DDL check warn! keyword not allowed with [table name]:text
WARNING:  LightDB DDL check warn! keyword not allowed with [column name]:id
WARNING:  LightDB DDL check warn! no primary key!
WARNING:  LightDB DDL check warn! no gmt_create or no gmt_modified!

Where 'text' is an nonstandard database name,So report a warning. When creating a table, the table name is not standard, the column name is not standard, does not contain the primary key, does not contain the creation time and the last modification time, and the fields are warned. Of course, there is the upper limit of 100 operation columns, which is not listed here.

By default, DDL syntax checking is not enabled,There are two ways to turn it on. Global modification by modifying the configuration file:

lightdb_best_practice_level = 'warn'
lightdb_best_practice_mode = 'all'

A value of 'warn' for lightdb_best_practice_level indicates that the error level is warning. A value of 'error' indicates an error. When an nonstandard statement is detected, an error is reported directly:

create database text;
ERROR:  LightDB DDL check error! keyword not allowed with [db name]:text

If lightdb_best_practice_mode is configured as' all ', it means to check 5 types at the same time. Of course, you can combine the checks you want to configure at will, for example:

lightdb_best_practice_mode = 'keyword_not_allowed,must_have_pk'

The above configuration means that only keywords and primary keys are checked.

Another configuration method is session level modification:

set lightdb_best_practice_level to 'warn'; 
set lightdb_best_practice_mode to 'all';

There are five kinds of fully open here, and there is another way:

set lightdb_best_practice_mode to 'keyword_not_allowed,bad_datatype,must_have_pk,must_have_gmt,no_more_100cols';

F.24.1.4. LightDB DML Specification Check

When you use DML statements, LightDB will first check whether your statements comply with the specification.

At present, there is one check, no paging check: Remind the user when the user SQL does not specify query criteria or paging. The usage of these checks is described in detail below.

  • no_paging check. Check is required in the following scenarios:

    • When the query statement has no where condition, no limit or no offset.

    • When the update statement has no where condition.

    • When the delete statement has no where condition.

How to use it? lightdb_best_practice_level and lightdb_best_practice_mode in the configuration file need to be configured.

By default, DML syntax checking is not enabled. Global modification by modifying the configuration file:

lightdb_best_practice_level = 'error'
lightdb_best_practice_mode = 'all'

A value of 'warn' for lightdb_best_practice_level indicates that the error level is warning. A value of 'error' indicates an error. When an nonstandard statement is detected, an error is reported directly:

select * from table1;
ERROR:  LightDB DML check error! select sql not where/limit/offset clause!

If lightdb_best_practice_mode is configured as' all ', it means to check all types at the same time. Of course, you can combine the checks you want to configure at will, for example:

lightdb_best_practice_mode = 'no_paging'

F.24.2. SQL interception

F.24.2.1. Overview

This feature will intercept risk sql that match the specified rule in lt_sql_inspect.secure_mode.

Only SQL that operates on the specified database will be blocked, using lt_sql_inspect.secure_enable_db to specify the database

You can define a time frame by lt_sql_inspect.secure_enable_time , and only in this time frame sql will be intercepted. The time frame can only be within a day.

lightdb_best_practice_level controls the log level when SQL is intercepted. default is 'warn'. A value of 'warn' for lightdb_best_practice_level indicates that the error level is warning. A value of 'error' indicates an error.

Refer to lt_sql_inspect.secure_mode for detail rules.

F.24.2.2. Configuration Parameters

lt_sql_inspect.secure_mode (String)

lt_sql_inspect.secure_mode specifies some SQL execution rules, such as intercepting SQL that meets the criteria. You can specify multiple rules, separated by commas. The following rules are supported: select_without_where, update_without_where, delete_without_where, no_paging, high_risk_ddl.

This setting can be changed within individual sessions.

  • select_without_where: Block a select query that queries only one table when it has no where condition, no limit and no offset.

  • update_without_where: Block a update query that queries only one table when it has no where condition, no limit and no offset.

  • delete_without_where: Block a delete query that queries only one table when it has no where condition, no limit and no offset.

  • no_paging: Equivalent to using select_without_where, update_without_where, and delete_without_where together.

  • high_risk_ddl: Block following ddl: drop table, truncate table, alter table add/drop column, alter table add/drop Constraint, alter table modify column(null/not null, default/no default, type).

lt_sql_inspect.secure_enable_time (String)

lt_sql_inspect.secure_enable_time specifies the time frame, and only in this time frame, sql will be intercepted.

Format is 'start_time - end_time'(example: '09:00 - 18:00' ).

This parameter can only be set in the lightdb.conf file or on the server command line, and can be reload.

lt_sql_inspect.secure_db (String)

lt_sql_inspect.secure_enable_db specifies the databases, and only on these databases, sql will be intercepted.

You can specify multiple databases, separated by commas.

This parameter can only be set in the lightdb.conf file or on the server command line, and can be reload.

F.24.2.3. Rules

F.24.2.3.1. select_without_where

DQL involving only one table without where, limit and offset will be blocked.

When querying table under pg_catalog, lt_catalog or ltcluster scheme, sql will not be blocked. Note that when the lightdb_best_practice_level is error, the query of table in other schemas(include tables created by extension) that meet the requirements will report errors.

lightdb@lt_test=# set lt_sql_inspect.secure_mode=select_without_where;
SET
lightdb@lt_test=# select * from t1;
ERROR:  lt_sql_inspect: SQL cannot execute, because a full table scan or modification may be performed on the table "t1"
lightdb@lt_test=# select * from (select * from t1);
ERROR:  lt_sql_inspect: SQL cannot execute, because a full table scan or modification may be performed on the table "t1"
lightdb@lt_test=# select * from t1 limit 1;
key1 | key2 
------+------
(0 rows)

lightdb@lt_test=# select * from t1 offset 1;
key1 | key2 
------+------
(0 rows)

     

DQL involving multi-table will not be blocked.

lightdb@lt_test=# select * from t1, t2;
key1 | key2 | key1 
------+------+------
(0 rows)

     

This also applies to the view just like a subquery.

lightdb@lt_test=# create view t1_v as select * from t1;
CREATE VIEW
lightdb@lt_test=# select * from t1_v;
ERROR:  lt_sql_inspect: SQL cannot execute, because a full table scan or modification may be performed on the table "t1"
lightdb@lt_test=# select * from t1_v where key1=1;
key1 | key2 
------+------
(0 rows)

     
F.24.2.3.2. update_without_where

Be similar to select_without_where, but for update statement.

lightdb@lt_test=# set lt_sql_inspect.secure_mode = 'update_without_where';
SET
lightdb@lt_test=# update t1 set key1=1;
ERROR:  lt_sql_inspect: SQL cannot execute, because a full table scan or modification may be performed on the table "t1"
lightdb@lt_test=# update t1 set key1=1 limit 1;
UPDATE 0
lightdb@lt_test=# update t1 set key1=1 offset 1;
UPDATE 0
lightdb@lt_test=# update t1 set key1=1 from t2 ;
UPDATE 0
lightdb@lt_test=# update t1 set key1=1 from t2 where t1.key1=t2.key1;
UPDATE 0
     
F.24.2.3.3. delete_without_where

Be similar to select_without_where, but for delete statement.

 
lightdb@lt_test=# set lt_sql_inspect.secure_mode = 'delete_without_where';
SET
lightdb@lt_test=# delete from t1 ;
ERROR:  lt_sql_inspect: SQL cannot execute, because a full table scan or modification may be performed on the table "t1"
lightdb@lt_test=# delete from t1 where key1=1;
DELETE 0
lightdb@lt_test=# delete from t1 limit 1;
DELETE 0
lightdb@lt_test=# delete from t1 offset 1;
DELETE 0
lightdb@lt_test=# delete from t1 using t2;
DELETE 0
lightdb@lt_test=# delete from t1 using t2 where t1.key1=t2.key1;
DELETE 0
     
F.24.2.3.4. no_paging

Equivalent to using select_without_where, update_without_where, and delete_without_where together.

lightdb@lt_test=# set lt_sql_inspect.secure_mode=no_paging;
SET
lightdb@lt_test=#   
ERROR:  lt_sql_inspect: SQL cannot execute, because a full table scan or modification may be performed on the table "t1"
lightdb@lt_test=# update t1 set key1=1;
ERROR:  lt_sql_inspect: SQL cannot execute, because a full table scan or modification may be performed on the table "t1"
lightdb@lt_test=# delete from t1 ;
ERROR:  lt_sql_inspect: SQL cannot execute, because a full table scan or modification may be performed on the table "t1"

     
F.24.2.3.5. high_risk_ddl

Currently the following ddl will be considered as a high risk ddl, and will be intercepted.

  • drop table

    lightdb@lt_test=# set lt_sql_inspect.secure_mode = high_risk_ddl;
    SET
    lightdb@lt_test=# drop table t1;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
           
  • truncate table

    lightdb@lt_test=# set lt_sql_inspect.secure_mode = high_risk_ddl;
    SET
    lightdb@lt_test=# truncate table t1;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
           
  • alter table add column

    lightdb@lt_test=# set lt_sql_inspect.secure_mode = high_risk_ddl;
    SET
    lightdb@lt_test=# alter table t1 add column key3 int;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
           
  • alter table drop column

    lightdb@lt_test=# set lt_sql_inspect.secure_mode = high_risk_ddl;
    SET
    lightdb@lt_test=# alter table t1 drop column key1;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
           
  • alter table add constraint

    lightdb@lt_test=# set lt_sql_inspect.secure_mode = high_risk_ddl;
    SET
    lightdb@lt_test=# alter table t1 add constraint t_uq unique(key1);
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
           
  • alter table drop constraint

    lightdb@lt_test=# set lt_sql_inspect.secure_mode = high_risk_ddl;
    SET
    lightdb@lt_test=# alter table t1 drop constraint t_uq;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
           
  • alter table modify/alter column(only the types in the following example will be blocked, including null/not-null, default/non-default, and type).

    lightdb@lt_test=# set lt_sql_inspect.secure_mode = high_risk_ddl;
    SET
    lightdb@lt_test=# alter table t1 modify key2 bigint;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
    lightdb@lt_test=#
    lightdb@lt_test=# alter table t1 alter  key2 type text;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
    lightdb@lt_test=#
    lightdb@lt_test=# alter table t1 modify key2 null;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
    lightdb@lt_test=#
    lightdb@lt_test=# alter table t1 alter  key2 drop not null;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
    lightdb@lt_test=#
    lightdb@lt_test=# alter table t1 modify key2 not null;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
    lightdb@lt_test=#
    lightdb@lt_test=# alter table t1 alter  key2 set not null;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
    lightdb@lt_test=#
    lightdb@lt_test=# alter table t1 modify key2 default 10;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
    lightdb@lt_test=#
    lightdb@lt_test=# alter table t1 alter  key2 set default 10;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
    lightdb@lt_test=#
    lightdb@lt_test=# alter table t1 alter  key2 drop default;
    ERROR:  lt_sql_inspect: SQL cannot execute, because it is a high risk ddl
    lightdb@lt_test=#