Recursive Promises Not Returning
I have a recursive function like so
function missingItemsPromise() {
return new Promise(resolve => {
if (missingItems == 0) {
console.log('resolves');
console.log(products);
return resolve();
} else {
page++;
url = getUrl(id, page);
http.get(url, function(xres) {
xres.setEncoding('utf8');
xres.on('data', function (xtraBody) {
console.log('calling');
var xtraJson = JSON.parse(xtraBody);
var xtraProducts = xtraJson['products'];
products = products.concat(xtraProducts);
productsLength = products.length;
missingItems = total - productsLength;
missingItemsPromise();
});
});
}
});
};
and I'm using it like
getInitial.
then(missingItemsPromise).
then(() => {
console.log('hello');
});
I'm noticing that the hello never returns because I suspect that I'm creating more than one promise on a recursive call but I"m unsure of how to return out of it.
How can I return each recursively created promise?
EDIT:
function missingItemsPromise() {
return new Promise(resolve => {
if (missingItems == 0) {
console.log('resolves');
return resolve();
} else {
page++;
url = getUrl(id, page);
http.get(url, function(xres) {
xres.setEncoding('utf8');
xres.on('data', function (xtraBody) {
console.log('calling');
var xtraJson = JSON.parse(xtraBody);
var xtraProducts = xtraJson['products'];
products = products.concat(xtraProducts);
productsLength = products.length;
missingItems = total - productsLength;
missingItemsPromise();
resolve();
});
});
}
});
};
results in
calling
hello <----notice here that it's already resolving once the first call resolve
is called
calling
calling
resolves
Recursion is a functional heritage and so using it with functional style yields the best results. That means writing functions that accept and operate on their inputs (rather than relying on external state) and return values (rather than relying on mutation or side effects).
Your program, on the other hand, calls functions without arguments, uses external state missingItems, products, productsLength, total, page and uses mutations like page++ and reassignments like products = ..., productsLength = ..., missingItems = .... We're gonna fix all of this!
I'm just going to blast thru this and hope it sets you on the right track. If you're stuck at the end, I link some other answers which explain the techniques used here in greater detail.
const getAllProducts = async (page = 0) =>
asyncUnfold
( async (next, done, [ res, nextPage ]) =>
res.products.length === 0
? done ()
: next ( res.products // value to add to output
, [ await getPage (nextPage), nextPage + 1 ] // next state
)
, [ await getPage (page), page + 1 ] // initial state
)
We introduce the getPage helper we used above
const getPage = async (page = 0, itemsPerPage = 5) =>
getProducts (page * itemsPerPage, itemsPerPage)
.then (res => res.json ())
Next, for the purposes of this demo, we introduce a fake getProducts function, and a fake DB where each product is simply a number. We also use delay to simulate real network delay.
In your real program, you just need to provide a getProducts function that can query products using offset and limit inputs
// fakes used for demonstration below
const getProducts = (offset = 0, limit = 1) =>
Promise.resolve
({ json: () =>
({ products: DB.slice (offset, offset + limit) })
})
.then (delay)
const delay = (x, ms = 250) =>
new Promise (r => setTimeout (r, ms, x))
const DB =
[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20
, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30
, 31, 32, 33
]
Below we demonstrate running the program. getAllProducts is a familiar async function which returns a Promise of its result. We chain a .then call so we can see all of the product pages output in the console
getAllProducts () .then (console.log, console.error)
// ~2 seconds later
// [ [ 1, 2, 3, 4, 5 ]
// , [ 6, 7, 8, 9, 10 ]
// , [ 11, 12, 13, 14, 15 ]
// , [ 16, 17, 18, 19, 20 ]
// , [ 21, 22, 23, 24, 25 ]
// , [ 26, 27, 28, 29, 30 ]
// , [ 31, 32, 33 ]
// ]
Instead of grouping products by page, it'd be nice if we could return all the products in a single array. We can modify getAllProducts slightly to achieve this
const concat = (xs, ys) =>
xs .concat (ys)
const concatAll = (arrays) =>
arrays .reduce (concat, [])
const getAllProducts = async (page = 0) =>
asyncUnfold
( ... )
.then (concatAll)
getAllProducts () .then (console.log, console.error)
// ~2 seconds later
// [ 1, 2, 3, 4, 5, 6, 7, ..., 31, 32, 33 ]Lastly, we introduce asyncUnfold
const asyncUnfold = async (f, initState) =>
f ( async (value, nextState) => [ value, ...await asyncUnfold (f, nextState) ]
, async () => []
, initState
)
Full program demonstration
// dependencies -------------------------------------------------
const asyncUnfold = async (f, initState) =>
f ( async (value, nextState) => [ value, ...await asyncUnfold (f, nextState) ]
, async () => []
, initState
)
const concat = (xs, ys) =>
xs .concat (ys)
const concatAll = (arrays) =>
arrays .reduce (concat, [])
// fakes --------------------------------------------------------
const getProducts = (offset = 0, limit = 1) =>
Promise.resolve
({ json: () =>
({ products: DB.slice (offset, offset + limit) })
})
.then (delay)
const delay = (x, ms = 250) =>
new Promise (r => setTimeout (r, ms, x))
const DB =
[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20
, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30
, 31, 32, 33
]
// actual program
const getAllProducts = async (page = 0) =>
asyncUnfold
( async (next, done, [ res, nextPage ]) =>
res.products.length === 0
? done ()
: next ( res.products
, [ await getPage (nextPage), nextPage + 1 ]
)
, [ await getPage (page), page + 1 ]
)
.then (concatAll)
const getPage = async (page = 0, itemsPerPage = 5) =>
getProducts (page * itemsPerPage, itemsPerPage)
.then (res => res.json ())
// demo ---------------------------------------------------------
getAllProducts ()
.then (console.log, console.error)
// ~2 seconds later
// [ 1, 2, 3, ..., 31, 32, 33 ]Other questions I've answered about recursion and promises
- Recursion call async func with promises gets Possible Unhandled Promise Rejection
- Recursive JS function with setTimeout
- Promise with recursion
Asynchrony and recursion are separate concepts. If you're struggling with asyncUnfold, it might help to first understand its synchronous counterpart unfold. These Q&A's may help distinguish the two.
- Functional way to create an array of numbers
- Loop until... with Ramda
- Node.js recursively list full path of files