View Issue Details

IDProjectCategoryView StatusLast Update
00022623 - Current Dev ListEnhancementpublic2018-06-13 21:47
ReporterWA9PIE 
Assigned ToK7ZCZ 
PrioritynormalSeverityfeatureReproducibilityN/A
Status feedbackResolutionopen 
Product Version6.4.0.787 
Target VersionFixed in Version 
Summary0002262: Add bulk callsign lookup in Logbook
DescriptionContext:

Users who import their logs from programs like N1MM complain that they have to open each individual QSO record and click the "callsign lookup button" in order to populate fields from the callsign lookup sources. They don't want to do this one-at-a-time.

The request is to enable bulk callsign lookup on selected records.

(This could actually apply to records imported from any third-party logging program. We hear it a lot from contesters who have used N1MM, N3FJP, Write Log... and so on. Point is to give folks a bulk callsign lookup option so they don't have to go one-by-one through several hundred QSOs to do lookups.)

The theory is that there is a "callsign lookup function". It should be the same function that is called when a callsign is looked up in the ALE. This is the same lookup function that should be used for this enhancement.
Steps To ReproduceHere's a use-case scenario:

User has some number of QSOs in the log where the worked station's fields have not been populated from the callsign lookup sources.
- User selects some number of QSOs in the log
- Right-click menu shows "Callsign Lookup" (see attached image below)
- Logbook sends each QSO out to the callsign lookup selections (same logic currently used when a single QSO is logged) and brings back data for that call... fills the fields as it would if it were a new QSO... then moves to the next selected QSO... repeats until all selected QSOs have completed.

This should probably run as a background process (in other words... not tie-up Logbook while this is happening).
Additional InformationWe 'could' take the opportunity to clean up the session problems with QRZ at this time. This is where the each lookup creates a new session key instead of maintaining it. Dunno.

We presently don't have any error messages that are presented when the callsign lookup fails - due to network issues, authentication... etc. We could add something like that... but I'm not requesting it at this time.
TagsNo tags attached.
ModuleLogbook
Sub-ModuleCall lookup
TestingNot Started

Relationships

related to 0001554 confirmed 1 - Backlog  Add function to use Callsign lookup on log import or initiate full log update 

Activities

WA9PIE

2018-03-05 23:55

administrator   ~0004445

If we do this, we should probably fix the login session key thing that Fred Lloyd at QRZ had raised recently.

WA9PIE

2018-03-11 22:31

administrator   ~0004477

Here's another request for the same thing.

batch-lookup.PNG (67,100 bytes)
batch-lookup.PNG (67,100 bytes)

WA9PIE

2018-04-25 21:49

administrator   ~0004888

Here is a mock-up picture of what this would look like after the user selects QSOs in the log and pulls up the right-click menu:

BulkCallsignLookup.png (85,796 bytes)
BulkCallsignLookup.png (85,796 bytes)

K7ZCZ

2018-04-26 14:18

manager   ~0004902

If we'd like to implement this feature, I think we should do some work to define how it behaves.

So far, we have a screenshot that is intended to show that we should be able to highlight multiple entries in a logbook view. (The program can already do that.)

The screenshot also shows that there's a new command in the "Lookup" tear-off menu from the context menu in the logbook listing view; that new command is named "Callsign Lookup".

That's probably not a great name, since there's already a "Callsign Lookup" command in the application (in the "Configure" tool bar button; and in the "More" tool bar button; and in the "Confure" tear-off of the "Tools" menu). The existing "Callsign Command" configures callsign lookup work.

It seems like "callsign lookup button" mentioned in the issue is the "Lookup" button on the main dialog inside the ALE, which uses the accounts and settings configured by the "Callsign Lookup" command to find details of a contact by their callsign.

We could proceed by looping over the selected items and looking them up; retrieving data from the callsign lookup sources, modifying each record with the returned data, and then writing the modified record back to the database.

Seems simple enough, but I have a few questions about the details:


  • How is progress represented to the user? A large number of look up operations can take a while, so we have to show the user progress somehow.

  • Is there a way for code to know if a record was previously looked up? If so, should this feature skip records that were previously retreived? Or does it just do the lookup operation again, overwriting was was there?

  • Is there a way for a user to know if a record was previously looked up? If I have 100 records in my logbook, do I always select them all and bulk lookup? Or do I know to select a few -- which ones?

  • That is, is there opportunity for a feature that looks up all records that haven't been looked up before? The user wouldn't even have to select them in the first place.

  • Can the user cancel? If so, how do we show them which work was completed and which wasn't?

  • How are errors handled? Do we stop at the first error? Are errors just ignored? Maybe a list is shown of the failing records; or they're left selected.

  • What can a user do about errors, anyway? Do they need to mark the record to not try again? Can't be printed or something? Say I run the bulk QRZ feature on 100 records. 98 work, 2 fail. What do I do next?



Writing a loop that performs the lookups isn't going to be too hard. Showing the results with the user and finding a good way for users to interact with the feature requires some thought and planning.

K7ZCZ

2018-04-26 14:22

manager   ~0004903

assigning for feedback about design

WA9PIE

2018-04-26 18:41

administrator   ~0004904

Last edited: 2018-05-06 15:44

View 2 revisions

Answers to the questions (de WA9PIE):

So far, we have a screenshot that is intended to show that we should be able to highlight multiple entries in a logbook view. (The program can already do that.)
>>The steps were updated with a bit more detail.

The screenshot also shows that there's a new command in the "Lookup" tear-off menu from the context menu in the logbook listing view; that new command is named "Callsign Lookup".
>>That's correct.

That's probably not a great name, since there's already a "Callsign Lookup" command in the application (in the "Configure" tool bar button; and in the "More" tool bar button; and in the "Confure" tear-off of the "Tools" menu). The existing "Callsign Command" configures callsign lookup work.
>>Customers refer to this as "Callsign Lookup." I'm open to other ideas. But we'd have to change this across all of Logbook and then convince folks to learn a new way to refer to it. (For example, it's under the Tools, Configure menu. But we could call it "Lookup.")

It seems like "callsign lookup button" mentioned in the issue is the "Lookup" button on the main dialog inside the ALE, which uses the accounts and settings configured by the "Callsign Lookup" command to find details of a contact by their callsign.
>>That's correct.

We could proceed by looping over the selected items and looking them up; retrieving data from the callsign lookup sources, modifying each record with the returned data, and then writing the modified record back to the database.
>>Right. But there's a method to the way the existing "function" is supposed to work... where it fills vacant fields in a particular order. This request assumes we'll use whatever function, routine, or code that is.

Seems simple enough, but I have a few questions about the details:

How is progress represented to the user? A large number of look up operations can take a while, so we have to show the user progress somehow.
>>I'm open to suggestions here. If a user does several hundred, they may not want a status bar sitting in their way of using the software. But maybe there's some way to provide status in an existing spot on Logbook in such a way that it all happens in the background. I'm open to suggestions here.

Is there a way for code to know if a record was previously looked up? If so, should this feature skip records that were previously retrieved? Or does it just do the lookup operation again, overwriting was was there?
>>The existing logic "doesn't care" whether or not the record has been looked up before. Maybe pop up a message that says, "Lookup will replace existing location information for these QSOs" might be good enough.

Is there a way for a user to know if a record was previously looked up? If I have 100 records in my logbook, do I always select them all and bulk lookup? Or do I know to select a few -- which ones?
>>There's no way to know if a record was previously looked up (same answer as to the previous question). This function (feature) will only lookup the records that were selected by the user.

That is, is there opportunity for a feature that looks up all records that haven't been looked up before? The user wouldn't even have to select them in the first place.

Can the user cancel? If so, how do we show them which work was completed and which wasn't?
>>Sure. If there's a way to do that. That would be great.

How are errors handled? Do we stop at the first error? Are errors just ignored? Maybe a list is shown of the failing records; or they're left selected.
>>I don't think there's any error handling in this now. We could certainly give them a list of the ones that errored out because there was no lookup info for them. (Probably bad calls anyway.)

What can a user do about errors, anyway? Do they need to mark the record to not try again? Can't be printed or something? Say I run the bulk QRZ feature on 100 records. 98 work, 2 fail. What do I do next?
>>The errors would basically be "call not found" or something. It's most likely because they entered the call wrong. So giving them a list of calls that failed would give them a list to go research.

K7ZCZ

2018-04-26 21:04

manager   ~0004906

Thanks for the answers. I think we need to be very crisp about how things will work before taking action; otherwise, nobody agrees that it does what was meant becuase what was meant wasn't very clear.

Making this process work in the background isn't too difficult, but the database code in the Logbook already suffers from severe concurrency issues. These issues are at the root of the DX Cluster crashing, for example. Progress visualization is straightforward; I think that a modeless dialog with a list that shows a callsign, a status, and a message should be adequate. These are sortable, and maybe there's a way to copy the list around.

I think it's important to understand if a previous lookup has been performed for two reasons. One is that it saves the user time by preventing work that's already been done from happening again. It might take two or three seconds to perform each lookup; that adds up quickly. The other concern is that information may be overwritten and changed; when a user doesn't expect that, it's a negative experience even if they had requested the update.

If we have no way to detect that a previous request has been succesful, then we can't avoid these scenarios and that doesn't seem like a very solid feature.

I don't know of code that tries to find or set fields in a particular order; I don't see mention of this in the product documentation, either. Is a description of this mechanism available? Otherwise, I'll have to take apart code and try to find it.

I'm sorry, but I don't understand the "same as previous" answer. The previous answer given was that the logic doesn't care, but logic never actually cares. I'm asking about the user, not the code, and I feel certain the user cares. If they didn't care, we wouldn't need to peform the lookups in the first place. Users don't want to spend time looking up the same information over and over again. They probably have the expectation that the software knows what has been done before; and if it doesn't, we should add that feature to support this one.

Finally, I want to know if this is the right feature. The product already has settings that work in the background to report QSOs to tracking websites. Would a feature that looks up newly added or imported QSOs be a better solution for whatever it is that customers are trying to solve?

If the problem the user is trying to solve is to fill missing information into the logbook, it seems natural that there should be a way to note which records haven't had a retreival attempt and service those without the user needing to manually find and select them.

WA9PIE

2018-05-10 15:45

administrator   ~0004990

While my flow-charting skills may not be sufficient, I've created a flow-chart for how I believe the existing callsign lookup function is supposed to work as-is. The purpose of this enhancement request is to enable this function to repeat itself for selected QSO records.

Good idea about using this at the time of import. That's certainly a great use-case.

But it's also needed for the folks who have existing records that are missing data... or where they failed to use the lookup at the time of import.

The feature of enabling newly added/imported records to be looked up in the background is a fine idea. However, the feedback I get from users is that they want to select a group of QSO records to invoke the lookup function for and have the fields filled.

Perhaps there should be an option to "Fill All" or "Fill Empty." Worth the conversation.

CallsignLookupFunctionFlow.pdf (387,621 bytes)

Issue History

Date Modified Username Field Change
2017-09-22 23:19 WA9PIE New Issue
2018-03-05 23:55 WA9PIE Note Added: 0004445
2018-03-05 23:57 WA9PIE Description Updated View Revisions
2018-03-11 22:31 WA9PIE File Added: batch-lookup.PNG
2018-03-11 22:31 WA9PIE Note Added: 0004477
2018-04-25 21:49 WA9PIE File Added: BulkCallsignLookup.png
2018-04-25 21:49 WA9PIE Note Added: 0004888
2018-04-25 22:16 WA9PIE Description Updated View Revisions
2018-04-25 22:16 WA9PIE Steps to Reproduce Updated View Revisions
2018-04-25 22:16 WA9PIE Additional Information Updated View Revisions
2018-04-26 14:18 K7ZCZ Note Added: 0004902
2018-04-26 14:22 K7ZCZ Assigned To => WA9PIE
2018-04-26 14:22 K7ZCZ Status new => feedback
2018-04-26 14:22 K7ZCZ Note Added: 0004903
2018-04-26 18:41 WA9PIE Note Added: 0004904
2018-04-26 18:44 WA9PIE Assigned To WA9PIE => K7ZCZ
2018-04-26 21:04 K7ZCZ Note Added: 0004906
2018-04-26 21:04 K7ZCZ Assigned To K7ZCZ => WA9PIE
2018-05-06 15:44 WA9PIE Note Edited: 0004904 View Revisions
2018-05-10 15:45 WA9PIE File Added: CallsignLookupFunctionFlow.pdf
2018-05-10 15:45 WA9PIE Note Added: 0004990
2018-05-10 15:46 WA9PIE Assigned To WA9PIE => K7ZCZ
2018-06-13 21:47 WA9PIE Relationship added related to 0001554