Where do the bullets come from on <li> elements?

The bullets are contained "within" the padding of <ul> element:

The padding is marked green and the margin orange:

Decreasing the padding shows that the bullets are "within" that padding:

Increasing the margin of the <ul> for example, shifts it right.

The list-style property controls the bullets themselves. Setting it to none will hide them. You would need to set the margin and padding to 0 if you want to get rid of the indent too.

ul
{
    list-style: none;
}

If you want to get rid of all margins / paddings and the bullet points, use this:

ul
{
    list-style: none;
    margin: 0;
    padding: 0;
}

Of course you can also apply bullets to other HTML controls:

div
{
    padding-left: 40px;
}
a
{
    display: list-item;
    list-style: disc;
}

<div>
    <a href="#">Item #1</a>
    <a href="#">Item #2</a>
    <a href="#">Item #3</a>
    <a href="#">Item #4</a>
</div>

Browsers typically have a "default stylesheet"—a set of CSS styles applying to specific HTML elements. Take a look at the Firefox default styles, Chrome's default styles, and IE's default styles.

Typically, a <ul> tag has the following default overridable CSS styles:

ul {
  display: block;
  list-style-type: disc;
  margin-before: 1em; /* equivalent to margin-top in most languages */
  margin-after: 1em;  /* equivalent to margin-bottom in most languages */
  margin-start: 0px;  /* equivalent to margin-left in LTR */
  margin-end: 0px;    /* equivalent to margin-right in LTR */
  padding-start: 40px;/* equivalent to padding-left in LTR*/
}

• list-style-type: disc causes a disc icon to appear next to the item.
• list-style-position and list-style-image are unset. Their defaults are outside and none, respectively. This means that the disc icon defined above will appear to the left of the li element (in most languages) and not interfere with the li display box itself.
• The margin and padding settings properly place the content.

An <li> tag as the following:

li {
  display: list-item;
}

• display: list-item is similar to display: block; and allows the separate list items to appear on different lines.

The round bullets for <li> elements is just the browser default. Just like it has a default font, fontsize, underline for links (blue), etc. To make sure you overwrite the browser defaults , use some css reset http://cssreset.com/ , or bootstrap css.

The style for <li> is defined by the <ul> or <ol> surounding it. for example

ul.circle {list-style-type: circle;}
ul.square {list-style-type: square;}
ol.upper-roman {list-style-type: upper-roman;}
ol.lower-alpha {list-style-type: lower-alpha;} 
ul.image{ list-style-image: url("//cdn.sstatic.net/stackoverflow/img/favicon.ico?v=6cd6089ee7f6");} 
ul.singleline { display:flex; list-style:none; } // css3 only

<ul class="circle">
    <li>one</li>
    <li>two</li>
    <li>three</li>
</ul>
<ul class="square">
    <li>one</li>
    <li>two</li>
    <li>three</li>
</ul>
<ol class="upper-roman">
    <li>one</li>
    <li>two</li>
    <li>three</li>
</ol>
<ol class="lower-alpha">
    <li>one</li>
    <li>two</li>
    <li>three</li>
</ol>
<ul class="image">
    <li>one</li>
    <li>two</li>
    <li>three</li>
</ul>
<ul class="singleline">
    <li>One </li>
    <li>Two </li>
    <li>Three </li>
</ul>

See this page https://css-tricks.com/almanac/properties/l/list-style/

The bullets come from ul elements. You can modify them using the properties list-style-type or list-style-image

I think the original question really was, "how can I add bullets to OTHER elements, since CSS supposedly lets you style anything (like p elements) to look like something else (like LI).

The answer seems to be, LI is a special case, AFAIK. Could be wrong. I tried adding <style="display: list-item; list-style-type:disc;"> on <p> elements, and didn't get bullets. (Tested with Chrome only).

You could actually add bullet characters (•) manually or programmatically to the start of each p element, and set margin and padding to look like a UL/LI block, I guess.