trepan2 is very slow (24 times slower than pdb) in stepping

[apteryks]

I've started experimenting with trepan2, which looks very promising :)

Unfortunately it appears to be very slow, too slow to use it as my regular debugging tool. The following is a small benchmark which basically just starts a cherrypy.

Save the following script somewhere, and install cherrypy (pip install cherrypy).

import time
TIME_START = time.time()

import cherrypy


if __name__ == '__main__':
    cherrypy.engine.start()
    cherrypy.engine.stop()
    cherrypy.engine.exit()
    print 'Execution took {}'.format(time.time() - TIME_START)

Now, run it with pdb, like this: pdb cherrypy_bench.py. Press 'c' to run until the end. I get: Execution took 0.215445995331

Doing the same, but using trepan2, I get: Execution took 5.29994106293

This is just a very simple example. With a real life scenario the time to just start a cherrypy or complex project could be 30 s or worst.

Is there something which could be done to speed it up?

[rocky]

The guts of trepan2 should be rewritten and revised in the area of stepping, It could use the same logic again as used in pdb, which comes from class Bdb for stepping and nexting. If you want to go at it, I encourage you to.

But I probably won't undertake doing that since for me trepan2 has been good enough for my needs for now and I have other stuff I've been working on.

The slowness comes from trepan2 insistence on accuracy over speed and a couple of features. The last time I looked, there were bugs in pdb when you did a next or step over. And I think in some cases continue can miss a breakpoint. pdb doesn't support gdb's finish (step out) if I recall correctly. There may be bugs in pdb's breakpoints when threads are involved too. I don't think most people notice these things, so probably one could live with the bugs in pdb/bdb.

More generally using the frame f_trace bit setting better could speed up tracing. This is something I think pdb could do better in as well. However the last time I looked at pdb and bdb the code base was a bit of a mess. Writing debuggers is generally a thankless task which is probably why the pdb code base is such a mess and no one takes credit for having written it or more actively maintains it.

trepan however is not slow if you modify your code enter the debugger, because there is no debugger before that. And that's what I have been using. Here, there is no debugger until you hit the first breakpoint.

If you come up with something, by all means submit a pull request.

[apteryks]

The guts of trepan2 should be rewritten and revised, It could use the
same logic again as used in pdb, which comes from class Bdb for
stepping and nexting. If you want to go at it, I encourage you to.

But I probably won't undertake doing that since for me trepan2 has
been good enough for my needs for now and I have other stuff I've been
working on.

This sounds like an interesting endeavour; my free time is well used on
many things already but I'll try having a look at it.

The slowness comes from trepan2 insistence on accuracy over speed and
a couple of features. The last time I looked, there were bugs in pdb
when you did a next or step over. And I think in some cases
continue can miss a breakpoint. pdb doesn't support gdb's finish
(step out) if I recall correctly.

Confusingly, 'return' in pdb seems to do what 'finish' does in gdb. I
use it often, for example when I step in somewhere by mistake.

The docs of pdb say [0]:

r(eturn) Continue execution until the current function returns.

More generally using the frame f_trace bit setting better could speed
up tracing. This is something I think pdb could do better in as
well. However the last time I looked at pdb and bdb the code base was
a bit of a mess. Writing debuggers is generally a thankless task which
is probably why the pdb code base is such a mess.

This is all very new to me (never heard of the f_trace bit). I'll have
to study the sources.

trepan however is not slow if you modify your code enter the debugger,
because there is no debugger before that. And that's what I have
been using. Here, there is no debugger until you hit the first
breakpoint.

I hadn't realized that trepan had the equivalent of import pdb; pdb.set_trace(), i.e., from trepan.api import debug; debug(). Thanks
for pointing that. My next question was "How can I get realgud
integration with this?" I couldn't find a solution "out-of-the-box", so
I hacked a function that I named "trepan2-delayed". Note that for the
following to work realgud needs this fix:
realgud/realgud#175 (comment) (thanks for
the prompt merge!). It's very similar to the trepan2 function, but it
starts the realgud process with "python" instead of "trepan2", while
setting up the hooks in Emacs to prepare for trepan2.

(with-eval-after-load 'python
  (load-library "realgud"))
(with-eval-after-load 'realgud
  (defun realgud:trepan2-delayed ()
    (interactive)
    (let* ((initial-debugger "python")
           (actual-debugger "trepan2")
           (cmd-str (trepan2-query-cmdline initial-debugger))
           (cmd-args (split-string-and-unquote cmd-str))
           ;; XXX: python gets registered as the interpreter rather than
           ;; a debugger, and the debugger position (nth 1) is missing:
           ;; the script-args takes its place.
           (parsed-args (trepan2-parse-cmd-args cmd-args))
           (script-args (nth 1 parsed-args))
           (script-name (car script-args))
           (parsed-cmd-args
            (cl-remove-if 'nil (realgud:flatten parsed-args))))
      (realgud:run-process actual-debugger script-name parsed-cmd-args
                           'realgud:trepan2-minibuffer-history)))
  (defalias 'trepan2-delayed 'realgud:trepan2-delayed))

Does this look reasonable, or is there something simpler/cleaner that would have
the same effect?

If you come up with something, by all means submit a pull request.

I'd be more than happy; in the meantime thank you for your answer and
for providing trepan/realgud! These are fine tools :)

[0] https://docs.python.org/2/library/pdb.html#debugger-commands

[rocky]

My next question was "How can I get realgud
integration with this?"

I have been running python in a shell and I run realgud-track-mode. But this isn't as good as the solution you give next.

I couldn't find a solution "out-of-the-box", so
I hacked a function that I named "trepan2-delayed". Note that for the
following to work realgud needs this fix:
realgud/realgud#175 (comment) (thanks for
the prompt merge!). It's very similar to the trepan2 function, but it
. starts the realgud process with "python" instead of "trepan2", while
setting up the hooks in Emacs to prepare for trepan2.

 (with-eval-after-load 'python
   (load-library "realgud"))
 (with-eval-after-load 'realgud ...

I tried this and it works pretty good. Some suggestions using it though. The default command to run shouldn't be trepan2 but python in this case.

[apteryks]

"R. Bernstein" <notifications@github.com> writes:

> My next question was "How can I get realgud > integration with this?" I have been running python in a shell and I run `realgud-track-mode`. But this isn't as good as the solution you give next.

OK, that's another good trick to know!

> I couldn't find a solution "out-of-the-box", so > I hacked a function that I named "trepan2-delayed". Note that for the > following to work realgud needs this fix: > realgud/realgud#175 (comment) (thanks for > the prompt merge!). It's very similar to the trepan2 function, but it . starts the realgud process with "python" instead of "trepan2", while > setting up the hooks in Emacs to prepare for trepan2. > ``` > (with-eval-after-load 'python > (load-library "realgud")) > (with-eval-after-load 'realgud ``` Tried this and it works pretty good. Some suggestions using it though. The default command to run shouldn't be `trepan2` but python in this case.

Right. It might be a bit confusing, but I believe that's what `trepan2-query-cmdline initial-debugger` does (initial-debugger is "python"). Or did you mean something else? Thanks.

[rocky]

I was probably picking up the unpatched code. I just tried and this works fine.

I've just added it to trepan2 and trepan3k, see realgud/realgud@a0af8eb . However really this should be turned into a macro to DRY that code, and then added across the board.

[apteryks]

Thank you! You're right, I guess it should be refactored in something closer to realgud:run-debugger in https://github.com/realgud/realgud/blob/master/realgud/common/run.el. That way it could be a common function realgud:run-debugger-delayed that would be used and specialized into realgud:trepan2-delayed, realgud:trepan3k-delayd, etc. For the Python debuggers the 'initial-debugger' could be made to use the python-shell-interpreter variable. I'm not very knowledgeable about macros yet; are there examples in the code base already?

[rocky]

https://github.com/realgud/realgud/blob/master/realgud/common/track-mode.el#L188-L203 might be of help.

Looking forward to a pull request. Thank you!

[rocky]

Also, perhaps the deferred calls could be in a separate history list, rather than realgud:trepan2-minibuffer-history. What do you think?