Yeah, I know this should be the basic question but I flip the page too fast and skip that chapter.
OK, the situation is the wire getting very hot when motor is running. Basically there are 2 sizes of wire I am using, thin and thick. From the image below the battery wire is thick one and the wire on controller is thinner one. Also, wires from motor are thinner, too and that's why I thought thinner wire might work as well? Now, please notice the wire after the connector from battery become thinner, this is the section getting hot when operation is going.