MySQL Subquery making query super slow
I've been struggling when it comes to optimizing the following query (Example 1):
SELECT `service`.*
FROM
(
SELECT `storeUser`.`storeId`
FROM `storeUser`
WHERE `storeUser`.`userId` = 1
UNION
SELECT `store`.`storeId`
FROM `companyUser`
INNER JOIN `store` ON `companyUser`.`companyId` = `store`.`companyId`
WHERE `companyUser`.`userId` = 1
UNION
SELECT `store`.`storeId`
FROM `accountUser`
INNER JOIN `company` ON `company`.`accountId` = `accountUser`.`accountId`
INNER JOIN `store` ON `company`.`companyId` = `store`.`companyId`
WHERE `accountUser`.`userId` = 1
) AS `storeUser`
INNER JOIN `service` ON `storeUser`.`storeId` = `service`.`storeId`
LIMIT 10;
The subquery should be returning something like "1","2","3,"4"
Anyway it's super slow and takes about 48 seconds to give a response, even though the subquery by itself, ran in a different console, takes about 0,0020ms to give results.
The same applies if I place the subquery inside an IN instead (Example 2):
SELECT `service`.*
FROM `service`
WHERE 1
AND `service`.`storeId` IN (
SELECT `storeUser`.`storeId` FROM `storeUser` WHERE `storeUser`.`userId` = 1
UNION
SELECT `store`.`storeId` FROM `companyUser`
INNER JOIN `store` ON `companyUser`.`companyId` = `store`.`companyId`
WHERE `companyUser`.`userId` = 1
UNION
SELECT `store`.`storeId`
FROM `accountUser`
INNER JOIN `company` ON `company`.`accountId` = `accountUser`.`accountId`
INNER JOIN `store` ON `company`.`companyId` = `store`.`companyId`
WHERE `accountUser`.`userId` = 1
)
LIMIT 10;
However if I simply put the values returned by that query, manually, it's basically instantly:
SELECT
`service`.*
FROM
`service`
WHERE 1
AND `service`.`storeId` IN (
"1", "2", "3", "4", "5"
)
LIMIT 10;
Important to mention that'd I've reviewed the indexes in the joins and everything seems to be in place, and the EXPLAIN [query] returns a filtered score of 100 for basically everything.
Edit:
Sorry for not providing enough information before, hope this can be more helpful:
MySQL 5.7,
Storage engine: InnoDB
EXPLAINs
1.) StoreUser
id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra
1 | SIMPLE | storeUser | NULL | ref | PRIMARY, storeUserUser | PRIMARY | 4 | const | 1 |100.00 | Using index
2.) CompanyUser
id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra
1 | SIMPLE | companyUser | NULL | ref | PRIMARY,companyUserCompany,companyUserUser | companyUserUser | 4 | const | 30 | 100.00 | Using index
1 | SIMPLE | store | NULL | ref | storeCompany | storeCompany | 4 | Table.companyUser.companyId | 5 | 100.00 | Using index
3.) AccountUser
id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra
1 | SIMPLE | accountUser | NULL | ref | PRIMARY,accountUserUser | accountUserUser | 4 | const | 1 | 100.00 | Using index
1 | SIMPLE | company | NULL | ref | PRIMARY,companyAccount | companyAccount | 4 | Table.accountUser.accountId | 305 | 100.00 | Using index
1 | SIMPLE | store | NULL | ref | storeCompany | storeCompany | 4 | Table.company.companyId | 5 | 100.00 | Using index
4.) Whole query (Example 2)
id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra
1 | PRIMARY | service | NULL | ALL | NULL | NULL | NULL | NULL | 2836046 | 100.00 | Using where
2 | DEPENDENT SUBQUERY | storeUser | NULL | eq_ref | PRIMARY,storeUserStore,storeUserUser | PRIMARY | 8 | const,func | 1 | 100.00 | Using index
3 | DEPENDENT UNION | store | NULL | eq_ref | PRIMARY,storeCompany | PRIMARY | 4 | func | 1 | 100.00 | NULL
3 | DEPENDENT UNION | companyUser | NULL | eq_ref | PRIMARY,companyUserCompany,companyUserUser | PRIMARY | 8 | const,Table.store.companyId | 1 | 100.00 | Using index
4 | DEPENDENT UNION | companyUser | NULL | ref | PRIMARY,accountUserUser | accountUserUser | 4 | const | 1 | 100.00 | Using index
4 | DEPENDENT UNION | store | NULL | eq_ref | PRIMARY,storeCompany | PRIMARY | 4 | func | 1 | 100.00 | NULL
4 | DEPENDENT UNION | company | NULL | eq_ref | PRIMARY,companyAccount | PRIMARY | 4 | Table.store.companyId | 1 | 100.00 | Using where
NULL | UNION RESULT | <union2,3,4>| NULL | ALL | NULL | NULL | NULL | NULL | NULL | NULL | Using temporary
Solution 1:
You didn't show us your indexes or EXPLAIN output, so all this is guesswork.
Clearly it's the subquery in your second example that's not optimized. That subquery is a UNION with three branches. The way you address performance trouble? Analyze and optimize each branch of the UNION separately.
You certainly need some better indexes, unless your database server is too small or misconfigured. That's very rare, so let's work on indexes.
The first branch is
SELECT storeUser.storeId
FROM storeUser
WHERE storeUser.userId = 1
This compound index covers that query. Try adding it. If you have a separate index on just userId, drop it when you add this one.
ALTER TABLE storeUser ADD INDEX userId_storeId (userId, storeId);
The second branch is
SELECT store.storeId
FROM companyUser
INNER JOIN store ON companyUser.companyId = store.companyId
WHERE companyUser.userId = 1
Subqueries with JOIN operations are a little tricker to optimize without access to EXPLAIN output, so this is guesswork. I guess these indexes will help, though. (Assuming you use InnoDB and the PK on store is storeId.)
ALTER TABLE companyUser ADD INDEX userId_companyId (userId, companyId);
ALTER TABLE store ADD INDEX companyId (companyId);
Similar analysis applies to the third branch of the UNION.
And, add this index. Your EXPLAIN points to it being missing, and so a full table scan of that large table being required.
ALTER TABLE service ADD INDEX storeId (storeId);
Again, helping you would be far easier if you showed us your table definitions with indexes. SHOW CREATE TABLE service; for example, would show us what we need for your service table. Pro tip when troubleshooting this kind of performance stuff always doublecheck your indexes. Ask me how I know that when you have a couple of hours to spare.
Pro tip Be obsessive about formatting your queries so they're readable. You, yourself a year from now, and your co-workers yet unborn need to read and reason about them. To my way of thinking that means skipping those silly backticks.
Solution 2:
Perhaps you need to rethink the schema. It seems like you need a table for "user" instead of, or in addition to, the 3 tables for different types of "users".
Meanwhile, these composite indexes are likely to help performance in either formulation:
storeUser: INDEX(storeId, userId)
storeUser: INDEX(userId, storeId)
service: INDEX(storeId)
store: INDEX(companyId, storeId)
companyUser: INDEX(userId, companyId)
company: INDEX(accountId, companyId)
accountUser: INDEX(userId, accounted)
When adding a composite index, DROP index(es) with the same leading columns. That is, when you have both INDEX(a) and INDEX(a,b), toss the former.
In particular, storeUser smells like a many-to-many mapping table. If so, see Many:many mapping for more discussion.
In general IN( SELECT ... ) does not optimize well, but you might find otherwise for your query.