Why are the hidden items in Fire Red/Leaf Green sometimes absent?

Solution 1:

Here is the logic. This is based off of a recreation of the game's code in C discovered by aphid, in the other answer to this question. As they said, the relevant code is mainly in /src/renewable_hidden_items.c. Obviously, this information is only valid if the people behind this project did a good job at recreating the game's code, but presumably they wouldn't have just made up an entire algorithm based on nothing.

1. Certain, but not all, of the hidden items in the game are marked as "renewable". Each renewable item is marked as either "common", "uncommon", or "rare" (or several of the above, see below).
2. All renewable hidden items are absent when the game starts.
3. Every time you take a step, a step counter increments. When it reaches 1500, you enter what we'll call "respawn mode". For reference, Celadon city is about 50 tiles across (west to east).
4. When in respawn mode, if you enter a zone that contains renewable hidden items, a global renewable item refresh is triggered. Your step counter is then reset to zero and you leave respawn mode.

The "global renewable item refresh" algorithm is:

1. Despawn all renewable hidden items in the entire game.
2. For each zone in the game that contains renewable items, do the following.
3. Generate a uniform random value from 0 to 99, inclusive.
4. If the value is from 90 to 99, respawn all rare items in this zone.
5. If the value is from 60 to 89, respawn all uncommon items in this zone.
6. Otherwise, respawn all common items in this zone.

A few points of emphasis, so there's no ambiguity:

1. Yes, when you enter a zone with renewable items, it triggers a global refresh across the entire game, not just that zone.
2. The refresh is only triggered when you enter a zone with renewable hidden items, not at any zone transition. Other zone transitions do not cause you to leave respawn mode and do not reset your step counter.
3. The random values generated per zone during a refresh are all independent.
4. However, the respawning of the items within a single zone are not, since they happen in batches: either all the rares, or all the uncommons, or all the commons.
5. It really is "all the rares", not "all the rares and anything less rare". Getting a number above 90 does not respawn the uncommons or the commons. However, in some zones items are simultaneously of several rarity levels, to basically emulate this "rare and under" behavior.
6. During a refresh, all items are despawned. This means once one spawns, if you don't go back and get it before the next respawn, there's a good chance it'll be gone by the time you get there.

Finally, the list of all renewable items in the game:

Note that because Berry Forest and Treasure Beach both have items of all three rarity levels, you're actually guaranteed to get items from them every 1500 steps. Also worth pointing out is the renewable Ether in the Underground Passage - I always thought Ethers were a finite resource in these games, but technically not (you just have to walk 15 thousand steps on average to get one).

Solution 2:

Someone has done part of the research necessary to answer this question already.

Take a look at the topic over here.

It appears that something in the game is dynamically affecting the flags for some hidden items (as there are no explicit references to these memory locations in the machine code for the game). Somewhere in the memory of the game is a list of boolean values that keeps track of which items currently are still on the map (see the topic above for details). These values are supposed to be set at 0x01 and remain so until the player picks up the item.

Suggestions of clearing the S.S. Anne or etc. are likely just incorrect; it's something else the user did that caused the items to appear.

Edit: Found the solution

It's random.

A lot of the time, people try and appear to succeed at finding patterns in things that are (pseudo)random. Instead, the game does the following: after the internal step counter reaches at least 1,500 steps a flag is set. If a new location is entered afterwards, items in this location are respawned with a fixed probability.

This means that if you walk through the game in a very efficient pattern, you will find fewer hidden items (as there will be a lower amount of spawns).

Look at the repository (a project that attempts to re-create code that compiles into the mainline pokemon games), specifically in /src/renewable_hidden_items.c,

{
    .mapGroup = MAP_GROUP(UNDERGROUND_PATH_EAST_WEST_TUNNEL),
    .mapNum = MAP_NUM(UNDERGROUND_PATH_EAST_WEST_TUNNEL),
    .rare = {
        HIDDEN_ITEM_UNDERGROUND_PATH_EAST_WEST_TUNNEL_ETHER,
        0xFF,
        0xFF,
        0xFF,
        0xFF,
        0xFF,
        0xFF,
        0xFF
    },
    .uncommon = {
        HIDDEN_ITEM_UNDERGROUND_PATH_EAST_WEST_TUNNEL_POTION,
        HIDDEN_ITEM_UNDERGROUND_PATH_EAST_WEST_TUNNEL_ANTIDOTE,
        HIDDEN_ITEM_UNDERGROUND_PATH_EAST_WEST_TUNNEL_PARALYZE_HEAL,
        HIDDEN_ITEM_UNDERGROUND_PATH_EAST_WEST_TUNNEL_AWAKENING,
        HIDDEN_ITEM_UNDERGROUND_PATH_EAST_WEST_TUNNEL_BURN_HEAL,
        HIDDEN_ITEM_UNDERGROUND_PATH_EAST_WEST_TUNNEL_ICE_HEAL,
        0xFF,
        0xFF
    },
    .common = {
        0xFF,
        0xFF,
        0xFF,
        0xFF,
        0xFF,
        0xFF,
        0xFF,
        0xFF
    }
},

Note: The North-South tunnel has an identical drop table.

There are 6 independent 70% chances of not getting the uncommon items to appear if the step counter ticked over before entering the underground tunnel. There's also one rare item that won't appear 90% of the time.

Thus there's a significant chance of p = .1058841 (10.6%) that you won't find anything. Try walking around within one of the routes connected to the underground for at least 1,500 steps, then re-entering it. See if the items can now be found.