High CPU utilization of JAVA process...what is going on?

On top of one kernel space thread LWP is running and user space threads are running on top of LWP.
It may be 1:1 or one: many. I just tried out finding thread ID (LWPID) of java process. The CPU utilization of java PID was too high. Usually I use following command
ps -Leo pid,ruser,vsz,rss,state,priority,nice,time,%cpu,comm,lwp,psr,nlwp
pid = Process ID
ruser = Real user ID
vsz = virtual memory size of the process in KiB (1024-byte units).
rss = resident set size, the non-swapped physical memory that a task has used (in kiloBytes).
state = Process state
priority = kernel scheduling priority.
nice = Nice value.
time = cumulative CPU time,
%cpu = cpu utilization of the process in "##.#" format. Currently, it is the CPU time used divided by the time the process has been running (cputime/realtime ratio), expressed as a percentage. It will not add up to 100% unless you are lucky. (alias pcpu).
comm = command name (only the executable name). Modifications to the command name will not be shown. A process marked is partly dead, waiting to be fully destroyed by its parent. The output in this column may contain spaces. (alias ucmd, ucomm). See also the args format keyword, the -f option, and the c option. When specified last, this column will extend to the edge of the display. If ps can not determine display width, as when output is redirected (piped) into a file or another command, the output width is undefined. (it may be 80, unlimited, determined by the TERM variable, and so on) The COLUMNS environment variable or --cols option may be used to exactly determine the width in this case. The w or -w option may be also be used to adjust width.
lwp and nlwp = lwp (light weight process, or thread) ID of the lwp being reported and number of lwp.
psr = processor that process is currently assigned to.
I have just kept it here for my own convenience. use man to get help of command.
As it was Weblogic server and java processes were owned by weblogic user, I used “| grep weblogic” and then filter with “ grep -v ' 0.0 '” to minimize output.
The above command gives us thread ID and CPU utilization of each thread. Many ways we can accomplish this task.
Use “ps –eo user,pid,%cpu,cmd | grep java” and then “ps u –Lp ” or “top –H –p .
Now we have to search thread id in process thread dump. Before that create thread dump using “kill -3 ” or JVM specific command (JDK/Jrokit).
#jstack > threadDump.txt
OR
#jrcmd print_threads > threadDump.txt
Search lwp ID as tid in thread dump that has taken from Jrokit JVM. Use hex value of lwp ID if java process uses jdk.

Process, child process and thread in Linux

Threading in Linux is (or was!!!) bit confusing.

Before coming to the main point, we need to have clear idea about process and thread. A process is an instance of a computer program that is being sequentially executed. A single process may contain several executable programs (threads) that work together concurrently to carry out the main function of the process. Threads share global data and address space with other threads running in the same process. On the other hand processes do not share address space. Child process has a own copy of data space (VM) of the parent process. Threads can directly communicate with other threads of its process; processes must use interprocess communication to communicate with child processes. Prior to Linux kernel 2.6, Linux OS was not good for JAVA multithreaded applications.
Introduction of NPTL offers performance improvement in the area of thread creation and destruction. With the new lightweight implementation, creating threads is fast and scalable.
Administrator use LWP id finding thread activity in java application. However, there are specific command in Solaris to get PID and associated lightweight process id.