How to encrypt usb pendrive/thumdrive/ that can be accessed from windows also?

For me, the easiest way to encrypt a thumbdrive that you can access from Windows and Ubuntu would be Veracrypt. It is available for both Windows and Ubuntu. I use it all the time.

https://www.veracrypt.fr

If you need to encrypt a container for storage in the cloud, consider Cryptomator. It is available for Windows, Ubuntu, and your phone/tablet.

https://cryptomator.org/

zip -e - no extra hardware or software - easy to use but not very secure

You get a simple and rather low security level encryption with an encrypted zip file, that can contain a number of files or directory trees. It works in Ubuntu and Windows as long as you use the standard encryption of the zip file.

There is zip in Ubuntu, and there are several tools in Windows, that are compatible (but if you use some 'enhanced encryption' in Windows, Ubuntu's zip will probably fail to decrypt it).

For all files in the current directory:

zip -e file.zip *

For all files in the current directory tree, 'recurse into subdirectories':

zip -er file.zip *

Unzip with

unzip file.zip

From man zip (in Ubuntu), we see that -e or --encrypt is the option to encrypt the archive.

   -e
   --encrypt
          Encrypt  the  contents  of the zip archive using a password which is
          entered on the terminal in response to a prompt (this  will  not  be
          echoed;  if  standard  error  is  not  a  tty, zip will exit with an
          error).  The password prompt is repeated to save the user from  typ‐
          ing errors.

but do not use the option -P or --password.

   -P password
   --password password
          Use password to encrypt zipfile entries (if any).  THIS IS INSECURE!
          Many multi-user operating systems provide ways for any user  to  see
          the current command line of any other user; even on stand-alone sys‐
          tems there is always the threat of over-the-shoulder peeking.  Stor‐
          ing the plaintext password as part of a command line in an automated
          script is even  worse.   Whenever  possible,  use  the  non-echoing,
          interactive prompt to enter passwords.  (And where security is truly
          important, use strong encryption such as Pretty Good Privacy instead
          of the relatively weak standard encryption provided by zipfile util‐
          ities.)

Higher level security

• Dedicated (already encrypted) USB pendrives already mentioned and linked to in the comment by @user68186.
• Systems with strong encryption
  • gpg is the gnu implementation of Pretty Good Privacy. It comes with Ubuntu because it is used to verify that the updates are correctly transferred, and you can use it to compress a file, for example a tar archive or an unencrypted zip file containing the files you want to transfer. There is a version for Windows, that you can install. (I installed gpg in Windows some years ago.)
  • Veracrypt, already mentioned and linked to in the answer by @Richard, is further developed from Truecrypt.
    • You have to install it both in Ubuntu and Windows
    • It is free, open source and has a good reputation
    • It can encrypt a whole partition with its file system and files.

I think Veracrypt is more convenient than gpg for this purpose, so if you want better security than with zip or similar tools, I would recommend Veracrypt or a dedicated (already encrypted) USB pendrive, that is compatible with linux.

7Zip AES-256

7Zip/P7Zip will encrypt per AES-256.

7Zip is available for Windows: https://www.7-zip.org/

P7Zip is available for Ubuntu: From Ubuntu Software Center and also by using:

sudo apt-get install p7zip-full

Neither worked for me producing an encrypted folder with drag and drop by themselves but installing both together let me create a 7Zip file as an encrypted folder requiring a password to open or when adding files.