emacs + realgud + trepan -> slow excecution during debug

[biocyberman]

I've been using pyCharm and like its debug features. However, they have quite confusing pricing policy, and more importantly I am using emacs more and more. So I want to get going with emacs-based python IDE. After searching here and there, I got the impression that trepan is the more feature-packed python debugger. So I want to give it a try. This lengthy paragraph is just to say I don't have much experience with trepan or working with python in emacs.

I use emacs remotely via SSH, and want to debug the python script on the remote server. However, compared to pdb, trepan is much slower after I do soft or hard restart. I had to kill python shell window because I could not wait.
I invoked trepan debugger like this: M-x realgud:trepan2. And then in (trepan2) buffer:

# because I know which line to debug before hand.
b 204
b 230
# modify some lines on the source code
# then hard restart 
restart
# code execution is very slow from here, even for lines that do importing.

So how do I imrpove this?

I have exactly the same issue and I think this happens because trepan uses an exhaustive execute-check-for-breakpoint strategy. Maybe someone can modify this behaviour by dynamically replacing the debugged code by self-breaking code? This is a problem for me and I've stopped to work in trepan-phycpp integration for the moment (currently near complete) because I'm busy with other planned parts. If rocky sends some feedback from where is the execution and test loop, I can attempt to solve this.

[rocky]

@melviso is roughly right, but I think the specifics for @biocyberman cause this misfeature in a different way. I'm guessing what's going on is that initially stopping is via a call to the debugger. In that scenario, execution is as fast Python itself because the debugger code hasn't even been imported!

However on a restart the debugger is called initially, and that is a lose. It is more of a lose for trepan than it is for say pdb which doesn't offer a hard restart. (Possibly for that reason, but I tend to think it was more of oversite than deliberateness). So in addition to what @melviso mentions, it's possible some sort of special restart would be desirable when the code has an explicit call to the debugger already in it.

But all of this is kind of off the top of my head. I could be wrong and often am. That's why I work on debuggers. I don't have the time to look I'll look now. When I do, I'll try to give more specifics that melviso might be able to work with.

Ok! Very thank you, rocky, I wait your news about, for doing my best effort! ;)

[rocky]

@melviso The code that I think is largely responsible for the slowness is is_stop_here
and possibly is_break_here

These deviate from the corresponding Python bdb code because there were bugs in that code with respect to implementing the finish command or "step until" and possibly the next or "step over" command. That is why I save the stack level and check that. However looking at the current code that might have
been addressed which mentions issue #13183.

There was also an optimization in pdb that also sped things up at the expense of accuracy. But I also don't see that in the current code though.

Given this, one simple fix is to ditch the tracer module I wrote altogether (which allowed for chaining of trace hooks) and use Bdb. If not that them probably cull from current bdb code and use that instead.

Something I had noticed fairly recently is that marks in the frame can be used to indicate stopping or not. Specifically this is a frame's f_trace. See
https://docs.python.org/2/library/inspect.html and that is used in Bdb.

If you have more questions, send email. I'm not sure this issue is the right place to track development of this fix. Afterwards we can always summarize what went on here.

[rocky]

@biocyberman : I sort of asked this implicitly, but since I haven't heard from you let me be more explicit. The first time you stop, that is done via call to debug() ?

For example:

from trepan.api import debug
# trepan is accessible but inactive.
# work, work, work...
debug() 

If so then a hard restart will clearly make this slower.

A possible solution is to add a flag on the restart command to skip invoking the debugger at the outside.

But before going down this route, I'd like to understand if this would address the situation you encounter.

hands-on work!

[biocyberman]

@rocky I have been busy switching to spacemacs. ~~I will come back to this when I debug python again ~~
Okay, I read your question again. I can answer it now.
The process that lead me to the problem is exactly discribed in the first post. I did not use import from trepan.api import debug and not invoking debug by debug() neither.

[diekhans]

Any progress on this issue? I am attempting to use trepan2 for the first time and have 60 second plus startup times due to the introspection. Soft restart gets the error

Traceback (most recent call last): File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/trepan/post_mortem.py", line 72, in post_mortem_excepthook post_mortem((exc_type, exc_value, exc_tb)) File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/trepan/post_mortem.py", line 155, in post_mortem sys.argv = list(dbg._program_sys_argv) AttributeError: Debugger instance has no attribute '_program_sys_argv'
so that isn't a soution.