[Marjolein Katsma]

tyrathect wrote on Nov 20 2007, 02:43 PM:

I'm not sure if I'm making myself clear about the feature I'm suggesting =^^= I'm imagining something like this: in the table data tab, right-clicking a row will bring up the usual popup menu but with one or two additional options, “Copy row data to insert row” and/or “Copy cell data to insert row” or something. Then I could edit the (copied) data in the insert row and hit the “Save changes” button which will at last physically insert the new row into the table.

It's certainly clear to me 😉 This is exactly what I'd like to see as well – it would save a lot of typing and navigating through fields – just copy row, and change the two fields or so that would really need to be different would be really helpful!

[Marjolein Katsma]

Thanks, Peter.

peterlaursen wrote on Nov 17 2007, 12:42 PM:

You are right – there is no change in this respect.

I still do not know if we can 'reinstantiate' PLINK if connection is lost between PLINK and he SSH Daemon.

I think this should be possible – in fact, I've been watching Process Explorer closely while SQLyog is doing something and I sometimes see an “extra” plink.exe appearing briefly near the end of a task, so 1) the program can obviously detect a missing connection (“MySQL server has gone away”) and 2) can clearly also start a new instance of plink.exe. To me that suggests a solution should be possible.

peterlaursen wrote:

mybe on Monday somebody else can tell if there has been considerations in this. The issue is recorded in our issue-tracker.

The 'Copy successful' problem (a different face of the same underlying problem?) is new though – and if anything more serious: if you don't go and check number of rows, you'd not even have a clue that the copy was not complete. I realised only later, after running into this with a few larger tables where the copy finished way too soon, that one of the first tables I copied – a small one – was also incomplete.

Please make sure this info is also added to the issue!

Thanks,

Marjolein

[Marjolein Katsma]

Alan Sawyer wrote on Sep 4 2007, 08:50 PM:

In a perfect world if I could find a command line SSH client for windows and eventually Mac, that would let me just shell off a ssh with the port, ip, username and password (maybe encrypted), and then have it connect and then I reference everything via localhost, that would be absolutely perfect, and I'm pay for it, well at least some reasonable amount for it.

And of course, I would want to make a one time payment for it, and not for each copy of my software

Have a look at Tunnelier – that at least has command line capabilities. Whether you can license it for such a use I don't know, but it couldn't hurt to ask 🙂

[Marjolein Katsma]

Simeon wrote on Sep 4 2007, 11:13 PM:

Obviously, complete removal is not a common approach, or we wouldn't have to re-install windows so regularly to remove left over 'junk' from software we have tried and then un-installed.

It's not a matter of “approach” but simply that (un)installers have no knowledge of what the application they install does, what it does it with, or what files it might possibly produce. Installers only know about putting files in positions defined by the application developer, and creating registry keys for them, to simplify it a bit. You can't remove what you don't know about, let alone make a sensible decision when you don't know what it is other than “a file I haven't installed”. 🙂 That's why it is polite to actually not remove what it doesn't know about, but tell the user something's been left behind. Unfortunately, not all (un)installers are as polite as that.

(I have the habit to clean up after programs I uninstall, but that's because I test a lot of software; I clean up in the Registry, too, as far as possible. And I don't reinstall Windows regularly – in fact I've done that only once during the 6-odd years I've been using this machine, as a result of a crash due to a hardware problem – not “junk” left behind, and it's still quite stable.)

Simeon wrote:

Failing that, there are a couple of solutions that I can think of off the top of my head-

A) Let the installer identify existing configuration files and ask if the user wants to retain, or overwrite them.

/:cool: shift any existing configuration files to a 'backup' folder somewhere, and notify the user so that the information can be 'salvaged' if required.

C) Only retain some parts of an existing configuration file (Only the connection details perhaps?)

Option C) is a bit hard as the application developer would have to create a script for the installer to run to edit a file… and not all installer generators can do such a thing. Option A and especially B are actually fairly common (and easy to specify for almost all installer generators) – I have several applications with a “backup” folder created by the installer where replaced files are put – that also makes a “rollback” by uninstalling the later version fairly easy.

But for updates/upgrades I have a habit of either installing in a new, parallel directory, or making a complete copy of a program directory before installing a new version over it – I don't trust all installers to be so clever so I create my own insurance. If I'm really suspicious, I may even export all relevant Registry keys before installation as well. 😉

[Marjolein Katsma]

peterlaursen wrote on Sep 4 2007, 01:02 PM:

Currently the same prefix will have to be used on every host!

I understand about “currently”. I was just giving a hint about a hopefully different future. 😛

[Marjolein Katsma]

Simeon wrote on Sep 3 2007, 11:32 PM:

Might need to look at adjusting the un-install process to clean up these left over folders too?

In general, an uninstall process removes only what the install process has created (using a log of the installation process) and does not remove any directories if they are not empty; some uninstallers are even polite enough to mention this.

Actually, this makes sense, since directories created by the installer may now hold data files that are still valuable to the user – the best option in this case would be to issue a warning what could not be removed. In addition, an (un)installer never has any notion of directories created by the running program (or by the user for the running program), so it could not possibly give even a warning about those, let alone remove them.

And what goes for directories and files in them usually also goes (more or less) for registry keys and values: what an (un)installer doesn't know about it cannot remove, though here a whole tree often will be removed when a top-level key is deleted, even if new subkeys had been created by the program.

[Marjolein Katsma]

peterlaursen wrote on Sep 4 2007, 03:18 PM:

OK .. it really looks like we should consider to re-initiate PLINK too.

For most of this week we will be busy 'merging' code from 4 contributors into the first 6.1 build. After that we will to schedule this!

Great! 😀

That said, it would also be a good idea, I think, to investigate what makes plink “go away” (or crash?). (As a programmer, I wouldn't be happy to know it sometimes “goes away” and not know why!) But it certainly would be more robust to to re-initiate plink.exe anyway, since at least the user may (intentionally or not) have killed the process.

Meanwhile, by now I have a gut feeling that plink “going away” (or crashing) is not dependent on idle time but somehow on how busy the system is. Maybe in such circumstances Windows takes away resources from plink that it assumes it still has, and it crashes as a result. Or something along those lines. My machine was really busy this morning with lots of software loaded and big backup jobs running, sometimes concurrently, with CPU maxing out at 100% for long periods.

Another thing to consider would be to close all handles used by a connection when that connection is closed. Same for threads (I see lots of those with in Process explorer). I also see a large number of Sections BaseNamedObjectsPlinkSharedErrorMsg in 6.05 (but not in 5.12 which apparently uses a different mechanism to communicate with its plink.exe) – I cannot imagine really 17 of those are needed with just two plink.exe processes doing nothing?

[Marjolein Katsma]

peterlaursen wrote on Jun 5 2007, 09:44 AM:

We have no such plans. We never encountered this request before.

I don't think it will be very hard to implement however, so if it really is a common need/request we can discuss.

I can see how this could be quite useful!

In many cases applications during installation allow the user to create a set of tables for it in an existing database, and let them choose a table prefix to avoid name clashes within that database. Now install the application again (say, two different forums running the same software, or two different wikis), and if you still have a single database (or just want to keep all wikis together in a single database) you just choose a different prefix. Then consider some migration scenarios (test to live, or one server to another) and it's easy to end up with two sets of tables with exactly the same structure where table names differ by prefix.

Regards,

[Marjolein Katsma]

At 10:37 I noted that one of v.5.12's three plinks has already gone away…

[Marjolein Katsma]

peterlaursen wrote on Sep 3 2007, 09:13 PM:

Just FYI: we cannot either consistently reproduce any different behaviour with the different versions in our own test.

Currently reconnection is attempted only once. We discussed:

1) providing a configuration option for user to set the # of reconnects

2) if first reconnect is not succesfull we could wait/sleep() for some time (say 200-ms or 500 ms) before trying again. The philosophy is that it will not make much sense to try again only a few milliseconds after failure. The remote host or or some network gear (routers, switches, transmitters) in the connection chain might need a little time …

Both of those would be good to have. Still, they'd mostly address symptoms, not the actual cause of the “gone away” problem – whatever that cause is.

peterlaursen wrote:

Also reconnection with SSH tunnel assumes that PLINK is alive. We do not re-instantiate PLINK. But I also do not think it is the problem that PLINK dies (except if it is explicitly killed by user, of course!).

Well, things are getting quite interesting here…

I remembered one of many the little tools I have installed on my machine: Process explorer (from Sysinternals). It's like Task manager on steroids, and one of the useful things is does is show how processes are related, by showing “parent” and “child” processes in a tree. For each, it can show an enormous amount of detail, like event, key, port, process, thread, etc. tree in the top pane, and either dlls or handles in the lower pane. In this case, the handles view is the most useful.

So yesterday I fired it up, and left that running as well as my two instances of SQLyog. What immediately struck me was that both had started two instances of plink.exe (matching the two consecutive popups of the “gone away” dialog, if it happens?). But what was more striking was that in the detailed list both had a number of [####] (with #### the original PID) mentioned as “process”, but my 6.05 instance (which also had been running longer) had many more of those than 5.12. I looked a few times at it yesterday, but didn't notice any change.

Now, I just had another look. 5.12 looks pretty much the same, with in its detailed process list still two and two plink.exe, just like I saw all the time yesterday. But now 6.05 is different: it has noting but listed (20!) and no plink.exe. I definitely did not kill those processes!

Now I can make a prediction: if I go to 5.12 and click on a new table, it will just show it to me; if I do the same in 6.05, it will give me the dreaded “gone away” dialog, twice: once for each plink.exe that's gone missing. So let's try… (I'm actually writing this before trying!)

v. 5.12:

• one click on new table while history tab is in view – nothing happens
• switch to data tab … long wait … and I'm getting the “gone away” dialog – once
• the data tab is empty
• looking at Process explorer: what has actually gone away is one of the two plink.exe processes that still were there just before!
• going back at the history tab, I see three new statements:
  Code:

  /*[07:50:16][ 50 ms]*/ show full fields from `travelblog`.`itinerary`

/*[07:50:17][ 0 ms]*/ Set names 'utf8'
/*[07:50:18][ 0 ms]*/ set sql_mode=''

• switching back to the data tab, I get another wait… and the “gone away” dialog again
• looking at Process explorer again… the second copy of plink.exe is still there (Task manager confirms this); in the detailed list, the original plink.exe item (recognizable by the PID which I noted) has now turned into a as well (3 of those now)

v.6.05:

• one click on new table while history tab is in view long wait … and I'm getting the “gone away” dialog – once
• switching to the data tab, I get another wait… and the “gone away” dialog again
• the history tab now shows one new statement for each attempt:
  Code:

  /*[07:59:14][ 131 ms]*/ show full fields from `travelblog`.`itinerary`

/*[08:01:44][ 0 ms]*/ show full fields from `travelblog`.`itinerary`

– the 0 ms for the second one is interesting!

• Process explorer still shows 20

So… my prediction was close, if not exact.

What's important to note is though that plink.exe can “go away”! Your assumption that it doesn't clearly isn't correct – and I definitely did not kill those processes. That leaves three possibilities:

• SQLyog is somehow killing them itself
• plink.exe decides to go away all by itself ('nothing to do here…')
• the operating system is killing them

I think the latter is actually more likely, and there may be a connection with “idle time” here as well; plink.exe going away by itself might fit, too – but remember I'm looking at two different versions of plink.exe now. I should add that I've had my machine pretty busy overnight (and right now) – it might be a case of Windows itself deciding to kill a child process when it's doing nothing anyway; but I'm not sure Windows is supposed to be doing something like that. Maybe plink.exe is simply crashing in some circumstances.

Whatever, the assumption that plink.exe is is always there is clearly false.

That should give you guys something to investigate. 😉

(If you don't have it yet, get Process explorer – it can show a ton of useful information.)

Meanwhile, I'll let both instances of SQLyog run, but in each close the connection and then re-start it. I should see two fresh copies of plink.exe for each right?

Wrong! For v.6.05 I see two (both in the tree and in the detailed list), but for 5.12 I see three in the tree but four in the detailed list; a little later there are only three in the detailed list as well – as if there was a fourth instance only briefly but it has already gone away! :huh: And 5.12 has two new in the process handles list while the two for the previous two copies of plink.exe are still there (apparently they are not closed when closing a connection!). 6.05 has one new – as if there briefly was a third copy of plink.exe that has gone missing already…

Task manager confirms: three of the older, two of the newer version of plink.exe running (easily recognized even there since they have different sizes resulting in different memory footprints).

Plenty of food for thought here, I think.

[Marjolein Katsma]

Marjolein Katsma wrote on Sep 3 2007, 08:33 AM:

I still have 5.12 installed as well (I'm weird that way ;)) which has a different version of plink.exe so I've now started that up, to see what happens – or does not happen…

Well, letting both versions (connected to the same server) sit idle for 9 or 10 hours and then trying a few clicks didn't show anything exciting – certainly no “Gone away” dialog. This is hard to reproduce – if it even has to do with sitting idle at all…

So, I tried something different. I followed a few actions in parallel on both clients (after the idle period) by a deliberate stop and start of MySQL server on the host, and then a few actions in parallel again. Still no “gone away” from either client version, but I do see an interesting difference between the two: it looks like v5.12 does a deliberate reconnect while v6.05 doesn't. (unless I'm interpreting the results incorrectly).

Here are the results:

v. 5.12 history

Code:

(after being idle since 09:16:20 today)
/*[18:54:04][ 50 ms]*/ Set names 'utf8'
/*[18:54:04][ 370 ms]*/ set sql_mode=''
/*[18:54:06][2153 ms]*/ show full fields from `travelblog`.`itinerary`
/*[18:54:06][ 20 ms]*/ show keys from `travelblog`.`itinerary`
/*[18:54:06][ 60 ms]*/ select * from `travelblog`.`itinerary` limit 0, 50
/*[19:18:18][ 20 ms]*/ show full fields from `travelblog`.`link`
/*[19:18:18][ 10 ms]*/ show keys from `travelblog`.`link`
/*[19:18:18][ 30 ms]*/ select * from `travelblog`.`link` limit 0, 50
/*[19:25:53][ 10 ms]*/ Set names 'utf8'
/*[19:25:53][ 10 ms]*/ set sql_mode=''
/*[19:25:53][ 311 ms]*/ show full fields from `travelblog`.`news`
/*[19:25:53][ 20 ms]*/ show keys from `travelblog`.`news`
/*[19:25:53][ 50 ms]*/ select * from `travelblog`.`news` limit 0, 50

v. 6.05 history

Code:

(last evening)
/*[21:39:05][ 10 ms]*/ Set names 'utf8'
/*[21:39:05][ 20 ms]*/ set sql_mode=''
…
(after being idle since 08:10:22 today)
/*[18:54:25][ 51 ms]*/ show full fields from `travelblog`.`itinerary`
/*[18:54:25][ 20 ms]*/ show keys from `travelblog`.`itinerary`
/*[18:54:25][ 10 ms]*/ show create table `travelblog`.`itinerary`
/*[18:54:35][ 30 ms]*/ show full fields from `travelblog`.`itinerary`
/*[18:54:35][ 20 ms]*/ show keys from `travelblog`.`itinerary`
/*[18:54:35][ 70 ms]*/ select * from `travelblog`.`itinerary` limit 0, 100
/*[19:19:11][ 20 ms]*/ show full fields from `travelblog`.`link`
/*[19:19:11][ 20 ms]*/ show keys from `travelblog`.`link`
/*[19:19:11][ 20 ms]*/ select * from `travelblog`.`link` limit 0, 100
/*[19:26:06][ 71 ms]*/ show full fields from `travelblog`.`news`
/*[19:26:06][ 10 ms]*/ show keys from `travelblog`.`news`
/*[19:26:06][ 20 ms]*/ show create table `travelblog`.`news`

mysql log .err

Code:

070903 19:25:14 [Note] /usr/sbin/mysqld: Normal shutdown

070903 19:25:16 InnoDB: Starting shutdown…
070903 19:25:17 InnoDB: Shutdown completed; log sequence number 0 43634
070903 19:25:17 [Note] /usr/sbin/mysqld: Shutdown complete

070903 19:25:17 mysqld ended

070903 19:25:25 mysqld started
070903 19:25:28 [Warning] Asked for 196608 thread stack, but got 126976
070903 19:25:28 InnoDB: Started; log sequence number 0 43634
070903 19:25:28 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.0.45-community' socket: '/var/lib/mysql/mysql.sock' port: 3306 MySQL Community Edition (GPL)

Another thing I could try is deliberately stopping/starting sshd on the server. I'll do that when I have a moment.

[Marjolein Katsma]

peterlaursen wrote on Sep 2 2007, 10:23 PM:

We asked Marjolein to compare different versions becuase we judged that she had a setup fit for such test, just as we have a similar test running at office since Friday. If this is an issue introduced in 6.0 or in late 5.x versions I hope Marjolein's information as well as similar information from our own test will reveal.

(…)

The culprit is if there is consistently reproducable difference between 6.x and various 5.x versions. Marjolein seems to have clear indications of this and this we take seriously!

By now I have seen similar behavior in 5.32 and 6.05 – both exhibit the “Gone away” problem. But I worked only for a bit with 5.22 combined with tunneling – so now I'm so not sure that really is different, the problem just didn't manifest itself, it's hard to reproduce on purpose… Those three versions have an identical plink.exe though, so if it's anything to do with plink, it's likely they behave the same, I just didn't see it with 5.22.

The difficulty is determining what is triggering the behavior, especially when it doesn't seem like it's anything server-side: you can only be sure it happens when it happens, but when it doesn't happen you cannot be sure it couldn't happen, it just happened not to happen. 😉

But I still have 5.12 installed as well (I'm weird that way ;)) which has a different version of plink.exe so I've now started that up, to see what happens – or does not happen…

[Marjolein Katsma]

c64audio wrote on Sep 2 2007, 06:18 PM:

I've been complaining about “Gone Away” messages in another thread, mostly concerned with its behaviour over SSH tunnelling. I got no satisfactory answer then, after they'd fixed structure sync.

Hi Chris,

I've been poking around the forum, of course, and noted the structure sync problem. but I understood that was either fixed or at least logged as a bug to be fixed.

The “Gone away” happens for me after I've been doing “nothing” for a long time (many hours, but it's hard to give a limit) – when using SSH tunneling. Are you saying you're seeing the same thing, or that it happened only with structure sync for you? Has your SQLyog maybe been sitting idle before (attempting to) start a structure sync?

I'm wondering if we could be seeing symptoms of the same underlying problem.

[Marjolein Katsma]

Marjolein Katsma wrote on Sep 1 2007, 09:11 PM:

Now, I can do a little digging to see if I can find if in the meantime either of the two services had stopped and auto-restarted. (Will report back if I find anything – I'm not familiar yet with where which logs are to be found so that may take me a while.)

Update – I found all the relevant log files:

• boot.log – nothing related to either sshd or mysqld for the relevant period of time
• messages – only relevant is a close of sshd session 4012 @ 21:06, which session was started

yesterday @ 8:55: that was me killing the session in SQLyog *after* MySQL had

“gone away”

• mysql's .err – shows mysqld has been running continuously since 2007-08-29 @ 22:55

So… neither mysqld nor sshd actually stopped on the server side during the relevant period; nothing on the server side (that I can find) that could cause this – and yet both clients (with the same plink.exe) report that “MySQL server has gone away”. Looks to me like plink.exe may be timing out after a (longish) period of idleness. But why can't it reconnect even then?

[Marjolein Katsma]

peterlaursen wrote on Sep 1 2007, 04:16 PM:

I think the best solution is that we should populate the OBJECTS tab for the connection itself!

Brilliant! It's completely blank now on the connection, and it makes sense.

Also, while typing this message I just found out that I cannot even copy anything from the window that pops up for the SHOW VARIABLES result – having the information on the Objects tab would solve that, too.

peterlaursen wrote:

We could display the connections manager information and the variables that are most important for SQLyog operation. We should also briefly explain how the affect SQLyog operations

I do not think 'uptime' is very relevant in the SQLyog context (we have MONyog for monitoring).

I considered the uptime mostly a gimmick (especially with the ticking seconds) in MySQL-Front. But server hostname or IP address is valuable – especially when you are working with multiple servers – and having it available at a glance (not even mouseclick). Maybe it could just be shown in the title bar?

peterlaursen wrote:

Those are most relevant I think:

Picking a few (going by what I see for my 5.0.45 server):

* server version

definitely – again, especially valuable if you are working with multiple servers, of different versions

* server default charset and collation

those, certainly, but I'd really like to see the whole set of character_set_* and collation_* values:

• character_set_client
• character_set_connection
• character_set_database
• character_set_filesystem
• character_set_results
• character_set_server
• character_set_system
• collation_connection
• collation_database
• collation_server

Those will be valuable when figuring out the odd problems that can come from working with combinations of different charsets and collations.

(BTW, the connection dialogs for 5.22 and 5.23 had a field for connection charset but it has disappeared in 6.05 – why? And since there's apparently a collation for the connection, wouldn't it be useful to be able to specify that, too? You might get some puzzling results if, say, your connection sorts “general” while the server does it “swedish”.)

(yes on the rest of the list)

and maybe

* # of active threads, idle / not idle count

Yes, definitely

Also:

All the various path values like:

• character_sets_dir
• datadir
• innodb_log_group_home_dir
• language
• pid_file
• slave_load_tmpdir
• socket
• tmpdir

They will be valuable as well when working with different systems, or a new system you're unfamiliar with. If you want to know if there is a socket, it helps if you know where to look. 😉

The various date and time formats could also be handy.

That's about what I can think of, based on the things I've been looking for when troubleshooting. But maybe others would have a need for other data…

Regards,

[Author]

Posts