删除1.7W条数据要30S-60S如何优化？

[BATTLERxANGE]

就执行一条SQL：
DELETE FROM TEMPCACHE WHERE TIMESTAMPDIFF(MINUTE,CRDATETIME,NOW())>=10
总共有3W条数据，删除的时候磁盘IO读写加起来飙到50M，执行要花将近1分钟，就删除大约1.7W条数据。
删完以后还会持续插入，10分钟后又有3W条数据然后继续执行删除，反复这个过程。
这个表其中有个字段是 MEDIUMBLOB 类型，存储的数据大了一点一般是10K-200K的样子。
CRDATETIME字段加了索引，为何这么慢？这因为索引碎片还是什么？怎么优化？

[ACMAIN_CHM]

这个没什么好办法，毕竟这么多记录。

[BATTLERxANGE]

引用 1 楼 ACMAIN_CHM 的回复:

这个没什么好办法，毕竟这么多记录。

才几W条数据也多吗？ 另外我发现即使条件不符合导致真正被删除的数据只有0条的时候，整个DELETE执行也要耗时几十秒，说明不是删除操作耗时，而是查找符合删除条件的行比较耗时，是索引没生效还是怎样？

[LongRui888]

DELETE FROM TEMPCACHE WHERE TIMESTAMPDIFF(MINUTE,CRDATETIME,NOW())>=10 改为 ： DELETE FROM TEMPCACHE WHERE CRDATETIME >= NOW() - INTERVAL 10 minute 然后 在 CRDATETIME上建个索引 试试

[BATTLERxANGE]

引用 3 楼 yupeigu 的回复:

DELETE FROM TEMPCACHE WHERE TIMESTAMPDIFF(MINUTE,CRDATETIME,NOW())>=10 改为 ： DELETE FROM TEMPCACHE WHERE CRDATETIME >= NOW() - INTERVAL 10 minute 然后 在 CRDATETIME上建个索引 试试

你给的语句好像有点问题？不好意思我对MYSQL不太熟，但我执行： select count(0) from TEMPCACHE WHERE CRDATETIME >= (NOW() - INTERVAL 10 minute); select count(0) from tempcache WHERE TIMESTAMPDIFF(MINUTE,CRDATETIME,NOW())>=10; 这两行SQL，发现结果不一致

[BATTLERxANGE]

环境是WIN2008 ，服务器配置：1G内存 SSD硬盘 这是我的表结构：

CREATE TABLE `tempcache` (
  `ID` char(36) NOT NULL,
  `Content` mediumblob,
  `CRDateTime` datetime NOT NULL,
  `Status` int(11) NOT NULL,
  PRIMARY KEY (`ID`),
  UNIQUE KEY `ID_UNIQUE` (`ID`),
  KEY `TIME_INDEX` (`CRDateTime`,`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

其中Content字段比较大，平均每条记录存储了30KB的数据。 这是我的my.ini配置： [client] no-beep # pipe # socket=mysql [mysql] default-character-set=utf8 # SERVER SECTION # ---------------------------------------------------------------------- # # The following options will be read by the MySQL Server. Make sure that # you have installed the server correctly (see above) so it reads this # file. # # server_type=3 [mysqld] # The TCP/IP Port the MySQL Server will listen on port=3306 # Path to installation directory. All paths are usually resolved relative to this. # basedir="C:/Program Files/MySQL/MySQL Server 5.6/" # The default character set that will be used when a new schema or table is # created and no character set is defined character-set-server=utf8 # The default storage engine that will be used when create new tables when default-storage-engine=INNODB # Set the SQL mode to strict sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION" # Enable Windows Authentication # plugin-load=authentication_windows.dll # General and Slow logging. log-output=FILE general-log=1 slow-query-log=1 slow_query_log_file="WIN-3JPGOV43BRO-slow.log" long_query_time=10 # Binary Logging. # log-bin # Error Logging. log-error="WIN-3JPGOV43BRO.err" max_connections = 500 max_user_connections = 500 query_cache_size=1M # The number of open tables for all threads. Increasing this value # increases the number of file descriptors that mysqld requires. # Therefore you have to make sure to set the amount of open files # allowed to at least 4096 in the variable "open-files-limit" in # section [mysqld_safe] table_open_cache=256 # Maximum size for internal (in-memory) temporary tables. If a table # grows larger than this value, it is automatically converted to disk # based table This limitation is for a single table. There can be many # of them. tmp_table_size=3M thread_cache_size=10 myisam_max_sort_file_size=100G myisam_sort_buffer_size=6M key_buffer_size=8M # Size of the buffer used for doing full table scans of MyISAM tables. # Allocated per thread, if a full scan is needed. read_buffer_size=0 read_rnd_buffer_size=0 # This buffer is allocated when MySQL needs to rebuild the index in # REPAIR, OPTIMZE, ALTER table statements as well as in LOAD DATA INFILE # into an empty table. It is allocated per thread so be careful with # large settings. sort_buffer_size=0 #*** INNODB Specific options *** # innodb_data_home_dir=0.0 # Use this option if you have a MySQL server with InnoDB support enabled # but you do not plan to use it. This will save memory and disk space # and speed up some things. # skip-innodb innodb_additional_mem_pool_size=2M innodb_flush_log_at_trx_commit=1 # The size of the buffer InnoDB uses for buffering log data. As soon as # it is full, InnoDB will have to flush it to disk. As it is flushed # once per second anyway, it does not make sense to have it very large # (even with long transactions). innodb_log_buffer_size=1M # InnoDB, unlike MyISAM, uses a buffer pool to cache both indexes and # row data. The bigger you set this the less disk I/O is needed to # access data in tables. On a dedicated database server you may set this # parameter up to 80% of the machine physical memory size. Do not set it # too large, though, because competition of the physical memory may # cause paging in the operating system. Note that on 32bit systems you # might be limited to 2-3.5G of user level memory per process, so do not # set it too high. innodb_buffer_pool_size=8M # Size of each log file in a log group. You should set the combined size # of log files to about 25%-100% of your buffer pool size to avoid # unneeded buffer pool flush activity on log file overwrite. However, # note that a larger logfile size will increase the time needed for the # recovery process. innodb_log_file_size=48M # Number of threads allowed inside the InnoDB kernel. The optimal value # depends highly on the application, hardware as well as the OS # scheduler properties. A too high value may lead to thread thrashing. innodb_thread_concurrency=2 # The increment size (in MB) for extending the size of an auto-extend InnoDB system tablespace file when it becomes full. innodb_autoextend_increment=64 # The number of regions that the InnoDB buffer pool is divided into. # For systems with buffer pools in the multi-gigabyte range, dividing the buffer pool into separate instances can improve concurrency, # by reducing contention as different threads read and write to cached pages. innodb_buffer_pool_instances=8 # Determines the number of threads that can enter InnoDB concurrently. innodb_concurrency_tickets=5000 # Specifies how long in milliseconds (ms) a block inserted into the old sublist must stay there after its first access before # it can be moved to the new sublist. innodb_old_blocks_time=1000 # It specifies the maximum number of .ibd files that MySQL can keep open at one time. The minimum value is 10. innodb_open_files=300 # When this variable is enabled, InnoDB updates statistics during metadata statements. innodb_stats_on_metadata=0 # When innodb_file_per_table is enabled (the default in 5.6.6 and higher), InnoDB stores the data and indexes for each newly created table # in a separate .ibd file, rather than in the system tablespace. innodb_file_per_table=1 # Use the following list of values: 0 for crc32, 1 for strict_crc32, 2 for innodb, 3 for strict_innodb, 4 for none, 5 for strict_none. innodb_checksum_algorithm=0 # The number of outstanding connection requests MySQL can have. # This option is useful when the main MySQL thread gets many connection requests in a very short time. # It then takes some time (although very little) for the main thread to check the connection and start a new thread. # The back_log value indicates how many requests can be stacked during this short time before MySQL momentarily # stops answering new requests. # You need to increase this only if you expect a large number of connections in a short period of time. back_log=80 # If this is set to a nonzero value, all tables are closed every flush_time seconds to free up resources and # synchronize unflushed data to disk. # This option is best used only on systems with minimal resources. flush_time=0 # The minimum size of the buffer that is used for plain index scans, range index scans, and joins that do not use # indexes and thus perform full table scans. join_buffer_size=256K # The maximum size of one packet or any generated or intermediate string, or any parameter sent by the # mysql_stmt_send_long_data() C API function. max_allowed_packet=4M # If more than this many successive connection requests from a host are interrupted without a successful connection, # the server blocks that host from performing further connections. max_connect_errors = 250 # Changes the number of file descriptors available to mysqld. # You should try increasing the value of this option if mysqld gives you the error "Too many open files". open_files_limit=4161 # Set the query cache type. 0 for OFF, 1 for ON and 2 for DEMAND. query_cache_type=1 # If you see many sort_merge_passes per second in SHOW GLOBAL STATUS output, you can consider increasing the # sort_buffer_size value to speed up ORDER BY or GROUP BY operations that cannot be improved with query optimization # or improved indexing. sort_buffer_size=256K # The number of table definitions (from .frm files) that can be stored in the definition cache. # If you use a large number of tables, you can create a large table definition cache to speed up opening of tables. # The table definition cache takes less space and does not use file descriptors, unlike the normal table cache. # The minimum and default values are both 400. table_definition_cache=400 # Specify the maximum size of a row-based binary log event, in bytes. # Rows are grouped into events smaller than this size if possible. The value should be a multiple of 256. binlog_row_event_max_size=8K # If the value of this variable is greater than 0, a replication slave synchronizes its master.info file to disk. # (using fdatasync()) after every sync_master_info events. sync_master_info=10000 # If the value of this variable is greater than 0, the MySQL server synchronizes its relay log to disk. # (using fdatasync()) after every sync_relay_log writes to the relay log. sync_relay_log=10000 # If the value of this variable is greater than 0, a replication slave synchronizes its relay-log.info file to disk. # (using fdatasync()) after every sync_relay_log_info transactions. sync_relay_log_info=10000 performance_schema_max_table_instances = 200 总的来说就是执行DELETE FROM TEMPCACHE WHERE TIMESTAMPDIFF(MINUTE,CRDATETIME,NOW())>=10 删除2W条数据，大概花了1.5分钟，是哪里没优化好吗？

[blizzf99]

内存换成8G~~

[ututuut]

可以试试关闭自动提交看，不知道会不会达到你的要求 mysql> set autocommit =0; Query OK, 0 rows affected (0.00 sec) mysql> delete frm try where ID>0; ERROR 1064 (42000): You have an error in your SQL corresponds to your MySQL server version for the r here ID>0' at line 1 mysql> delete from try where ID>0; ERROR 1146 (42S02): Table 'try.try' doesn't exist mysql> delete from tempcache where ID>0; Query OK, 1328 rows affected (0.01 sec)

[--船长--]

兄弟，锁表了，3楼正解