+51
Completed
Azure DevOps integration (like GitHub, Bitbucket)
Visual Studio Team Services is a popular and common tool among .net developers.
Although SmartGit can clone / pull ... from the Git repo, it is lacking proper integration similar to Github.
For example it would be desired to be able to do Pull Requests from within SmartGit, rather than jumping to another tool.
Customer support service by UserEcho
you can now change this to Azure DevOps Server since tfs is renamed by microsoft
Thanks for pointing out!
I could work with simple azure DevOps repositories without any problem. After configuring pre-merge validation, I needed to create Pull Requests to send my changes to the repository. It would be great, if my favourite tool - SmartGit could do it.
Seeing PRs would be a great thing ...
how many votes does this feature request need before smartgit picks up development efforts for this?
The main feature I'd like to see to see for the initial integration of Azure DevOps and SmartGit is to be able to authenticate to an Azure DevOps repo from within SmartGit using an Azure DevOps Personal Access Token.
Absolutely! It's what I'm waiting 4 years. :)
@all, is the authentication with a Personal Access Token the main reason for your upvotes?
no. Initially, the aim was to create pullrequests from within smart-git and i would also like to link bug, backlogs and task with a preview. here is an example of an azure dev ops input which displays a list of suggestions after the hashtag character
No,
As the original request posted 4 years ago we stated that our main need is to be able to do Pull Requests similar to how it is possible with Github and Bitbucket.
It is sad that after 4 years and this is still not available, it's not like Azure DevOps is unpopular and rarely used.
Of course we couldn't wait around for SmartGit to deliver on this, so we wrote our own external integration, that utilizes DevOps APIs and Jira APIs to create the necessary PR and link to a Jira Ticket, This is still a half baked solution and far from a seamless integration that a tool like SmartGit should provide.
Yes for me.
After some reflection I realized that I actually haven’t had authentication problems with SmartGit and Azure DevOps for quite a while, so I spent some time experimenting to figure out just how SmartGit is authenticating to my repos in Azure DevOps. I ultimately determined that it uses Personal Access Tokens via the Git Credential Manager. So explicit support of PATs by SmartGit probably isn’t a critical feature after all. When you install Git (on Windows at least), you have the option to install the Git Credential Manager as well. As long as you do that you should have smooth sailing.
As far as advanced features like pull request integration goes, I don't mind managing pull requests in Azure DevOps, but SmartGit integration would be nice. It's worth nothing that neither Visual Studio nor Visual Studio Code currently provide support for Azure DevOps pull requests. As far as I can tell, there’s one extension in beta that was written by somebody as a pastime, and Microsoft had previously provided extensions for functionality in VS and VS Code that supported pull requests but they’ve been or are being deprecated and have not yet been replaced. So Pull Request integration could be an opportunity for SmartGit.
Azure DevOps isn’t as popular as GitHub and maybe not even as popular as GitLab, but a lot of people do use it, so adding Azure DevOps support could help increase the SmartGit user base. I generally prefer using SmartGit over Visual Studio or VS Code for repository management.
21.1 Preview build 17068 contains an integration comparable to GitHub. Some questions/notes:
To give it a try, get 21.1 Preview from https://www.syntevo.com/smartgit/preview/ and invoke Help|Check for Latest Build.
Feedback is much appreciated!
Awesome!. Thanks Marc! I'll play around with this over the weekend.
@Marc - I downloaded v 21.1 preview 4 and took a look at changelog.txt. I couldn't find any mention of Azure DevOps in the file - searches for "azure" or "devops" returned zero results....
@Marc, this is promising, but
Same here, I'm running 21.1 preview 3 #17068, I don't see anything in the menus or elsewhere that are PR related.
The changelog.txt is only updated for an official preview release, not for each intermediate build. In the preferences, hosting providers section you can find Azure DevOps.
Thanks Marc. I've installed the preview but I don't see Azure DevOps in the hosting provider list. Also, the hosting providers are listed in a different order than in your screenshot. I've also included a screenshot of my OS specs.
Thanks.
As pointed out by hrant, you will have to get the latest build: Help|Check for Latest Build (not Help|Check for New Version!)
@Martillo make sure you check for updates and installed build is 17068, from the looks of the title bar yours is going to expire on April 29th, so it was probably last updated on March 29th which wouldn't include that build
As for our case.
I found few issues with Hosting providers Azure DevOps that are blocking.
1- If enter Personal Access Token as it alludes to that I can, it fails, and yes the token is valid.
What does the Account field represent? is it a label to name the account or is it actually my account email?
Or is it the organization name?
I tried all possibilities and all failed.
If I try to Generate Token instead, I get redirected to a browser and get an Authorization Code? however this is not clear as to what organization it is using?
We have multiple Azure DevOps Organizations.
If I enter one of the Organization names in the Account section, The SmartGit gets stuck and the only way out of it is to force kill it through task manager
> What does the Account field represent? is it a label to name the account or is it actually my account email? Or is it the organization name?
It should be the organization name. It's the same what will be used as prefix for your URLs. I guess we should rename it to "Organization".
> If I try to Generate Token instead, I get redirected to a browser and get an Authorization Code? however this is not clear as to what organization it is using?
Good question. I guess the one(s) for which you are logged in. Frankly, I do not yet understand the relationship between users (accounts?) and organizations.
> If I enter one of the Organization names in the Account section, The SmartGit gets stuck and the only way out of it is to force kill it through task manager
You can see all sent REST API requests in logs/log.txt.0 in the Settings directory. Now to start investigating these different problems:
(1) please get 21.1 portable bundle
(2) have a fresh setup and update to the latest build
(3) reproduce a single problem
(4) check logs/log.txt.0 yourself, maybe you will immediately see what might be wrong with the requests; if so, please let me know; if not, send me compressed logs/log.txt.0 to smartgit@syntevo.com
(5) get rid of the entire .settings directory of the portable bundle
(6) Restart at (2) with the next problem
I installed the latest build and experienced the same issues as reported by hrant (although SmartGit never froze up on me).
In Azure DevOps, an Organization is the overall container for a collection of projects. Each project contains agile/scrum/kanban boards, pipelines, repositories, artifacts and more. Users are given access to an organization and personal access tokens are assigned to a user at the organization level. Azure DevOps admins then assign users specific permissions within the org. Any Microsoft personal, work or school account or a GitHub account can be used to create a user at the Org level in Azure DevOps. One or more Azure DevOps organizations can be connected to an Azure Active Directory. Hope that helps...
BTW - the text of this error message could use some tweaking...
Thanks, I'm going to fix the error message. Still, I can't reproduce your problem. Please send logs, as explained in my reply to hrant.
Hi Marc,
> Good question. I guess the one(s) for which you are logged in. Frankly, I
do not yet understand the relationship between users (accounts?) and
organizations.
Here it gets worse, a user logged in can be a member of multiple organizations, I do happen to be part of multiple organizations, with a single login, hence why I asked which organization is picked up.
I think this is very critical in the implementation as the REST APIs are all based on the organization.
I found this stack overflow information on how to get a list of organizations
https://stackoverflow.com/questions/54762368/get-all-organizations-in-azure-devops-using-rest-api
As suggested I downloaded the latest portable, and upgraded to the latest preview
logged out of Azure DevOps
when I attempt to add the Hosting Provider, the app made a rest call (extracted from the logs) to
2m21s129 (2021-04-02 07:39:56) [main] INFO q.browserLauncher - Launching command='["rundll32", "url.dll,FileProtocolHandler", "https://app.vssps.visualstudio.com/oauth2/authorize?client_id=REDACTED&response_type=Assertion&state=nothing&scope=vso.code_write&redirect_uri=https://www.syntevo.com/smartgit/azuredevops-callback"]' in 'C:\Users\hrant'
And I got this on the browser
It doesn't look like the logs are capturing the request headers.
Thanks
This call is not actually sent from SmartGit, but it's an URL which is passed to your browser. A "Request Too Long" is definitely not expected here. The URL looks identical for me (client_id is a constant) and it will finally result in opening the callback URL:
https://www.syntevo.com/smartgit/azuredevops-callback
What SmartGit is doing here is explained at:
https://docs.microsoft.com/en-us/azure/devops/integrate/get-started/authentication/oauth?view=azure-devops#authorize-your-app
Could this be some firewall issue in your company (network)?
Unfortunately I'm swamped with work right now and won't have time to do the more extensive testing that Marc suggested until this weekend, and only then if the fates smile on me. However, I was able to spend a little time fooling around with it today...
I had zero luck setting up Azure DevOps as a hosting provider using Personal Access tokens, but clicking the Generate Token button and then using the token generated at https://www.syntevo.com/smartgit/azuredevops-callback/ worked! I was able to create two separate Azure DevOps hosting provider accounts for two Azure DevOps orgs that I authenticate with using the account that I'm currently logged into my Windows workstation with. I was even able to give the Accounts more descriptive names in the SmartGit interface (one note: these accounts disappeared when I closed and then re-opened the Preferences dialog. I recreated the accounts and was unable to reproduce their disappearing act after that).
The SmartGit App Authorizations show up in the User Settings section of Azure DevOps. However, when I tried to generate a token for a third Azure DevOps org where I'm an administrator via a Microsoft account that is NOT the one I'm logged into my workstation with, SmartGit returned a 401 Forbidden error.
Hi Marc,
Here's more details
> This call is not actually sent from SmartGit, but it's an URL which is
passed to your browser. A "Request Too Long" is definitely not expected
here. The URL looks identical for me (client_id is a constant) and it
will finally result in opening the callback URL:
Correct, it turns out that smartgit passing url to browser is running into an issue and concatenating multiple times.
here's how the urldecoded version looks like what is actually passed to the browser.
```
https://app.vssps.visualstudio.com/_signin?realm=app.vssps.visualstudio.com&reply_to=https://app.vssps.visualstudio.com/_signin?realm=app.vssps.visualstudio.com&redirect=1&force=1&context=REDACTED&reply_to=https%3A%2F%2Fapp.vssps.visualstudio.com%2F_signin%3Frealm%3Dapp.vssps.visualstudio.com%26redirect%3D1%26force%3D1%26context%3DRE...
```
if I copy the url shown in the dialog and paste that in the browser, I get a proper login dialog, and when I paste back the Authorization code, the Account field is filled with one of our organizations.
If I repeat the process and this time I manually change the Account to my other organization (same account credentials by the way)
Then Smartgit falls into an infinite loop when doing the REST call to
https://app.vssps.visualstudio.com/_apis/profile/profiles/me
And I have to kill it to get out of it.
Here's the logs portion
```
Sending GET 'https://app.vssps.visualstudio.com/_apis/profile/profiles/me'
3m 5s159 (2021-04-02 16:20:38) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=0'
3m 5s386 (2021-04-02 16:20:38) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=50'
3m 5s626 (2021-04-02 16:20:39) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=100'
3m 5s881 (2021-04-02 16:20:39) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=150'
3m 6s 38 (2021-04-02 16:20:39) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=200'
3m 6s213 (2021-04-02 16:20:39) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=250'
3m 6s439 (2021-04-02 16:20:40) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=300'
3m 6s663 (2021-04-02 16:20:40) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=350'
3m 6s826 (2021-04-02 16:20:40) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=400'
3m 7s 32 (2021-04-02 16:20:40) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=450'
...
7m23s911 (2021-04-02 16:24:57) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=64300'
7m24s 69 (2021-04-02 16:24:57) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=64350'
7m24s229 (2021-04-02 16:24:57) [ThreadPool-6 (smartgit.TM$$Lambda$849/0x0000000100730840)] INFO q.json - Sending GET 'https://dev.azure.com/REDACTED/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=64400'
```
Now if we forget about the second org, and look into the first one that properly got added, I still don't see anywhere any traces of pull request.
Hope this helps.
Thanks
> Correct, it turns out that smartgit passing url to browser is running into an issue and concatenating multiple times.
> here's how the urldecoded version looks like what is actually passed to the browser.
That's interesting. I'm on Windows 10 and don't have any troubles with passing over url-decoded version to rundll32 (the partial duplication is expected and required). From command line, can you please try to run following commands and let me know which ones of them succeed?
rundll32 url.dll,FileProtocolHandler "https://app.vssps.visualstudio.com/oauth2/authorize?client_id=8596D322-85D7-49ED-8103-77ABE77BD190&response_type=Assertion&state=nothing&scope=vso.code_write&redirect_uri=https://www.syntevo.com/smartgit/azuredevops-callback"and:
rundll32 url.dll,FileProtocolHandler "https://app.vssps.visualstudio.com/oauth2/authorize?client_id=8596D322-85D7-49ED-8103-77ABE77BD190&response_type=Assertion&state=nothing&scope=vso.code_write&redirect_uri=https%3A%2F%2Fwww.syntevo.com%2Fsmartgit%2Fazuredevops-callback"> if I copy the url shown in the dialog and paste that in the browser, I get a proper login dialog, and when I paste back the
> Authorization code, the Account field is filled with one of our organizations.
That's good. This is the result of https://app.vssps.visualstudio.com/_apis/accounts callback. When invoking this URL in your browser can you see multiple accounts there? Does that match the organizations you are connecting to?
> Now if we forget about the second org, and look into the first one that properly got added, I still don't see anywhere any traces of pull request.
Does the Branches view now show the Azure logo? Let's first concentrate to get this working for the organization which Generate Token suggests. Also let's switch to email. Please send clean logs for the seemingly successful authentication for this organization to smartgit@syntevo.com:
https://www.syntevo.com/doc/display/SG/Debugging#Debugging-Generaladvices/preparations
Also as I previously reported and now confirmed by Martillo
Entering Personal Access Token does not work, I can't see how it would work as it never asks for username, which would be needed if you want to authenticate with Personal Access Token.
I haven't had a chance yet to get set up to capture logs - may later tonight if I'm lucky. I did a quick test invoking those rndlle32 invocations at the command line on Windows 10 and they both took me to the Azure DevOps Authorization page at https://www.syntevo.com/smartgit/azuredevops-callback. Invoking https://app.vssps.visualstudio.com/_apis/accounts in my browser returns a JSON list of Azure DevOps orgs that match my current login. I see the Azure logo in the branches view for a repo in one of the orgs I was able to configure in the Hosting Providers dialog.
That looks good! Currently, SmartGit is populating the "Account" field with the first account from this list. Maybe it should better offer a list to chose from?
Did you also try to click the Generate Token button instead of copy&pasting the URL to the browser directly? It will invoke rundll32 exactly you did, hence I would expect this to work, too.
Regarding pull requests, the Branches view doesn't show any of them, but it also doesn't denote any errors in the Azure logo. Is it correct that the repository doesn't contain any pull requests?
@Marc,
Sorry for the delayed response, didn't realize that you had responded, just realized that your response was nested higher up.
Confusing if you ask me, as I would need to go all over my past posts to see if there were any further comments / responses on it.
In any case, here's more feedback.
1- Both rundll commands work with all browsers except Firefox, it appears that this is specific to my instance of Firefox and that also worked on another system, so you can ignore that issue.
2- Digging further on why I was not seeing the Azure logo on branches, it dawned on me that our organization still uses the older visualstudio domain format
`https://org.visualstudio.com` instead of `https://dev.azure.com/org`
https://docs.microsoft.com/en-us/azure/devops/release-notes/2018/sep-10-azure-devops-launch#administration
Switching one of the repos to the new domain format, I'm able to see the Azure logo on the Branches, however I'm also seeing this
> 2- Digging further on why I was not seeing the Azure logo on branches, it dawned on me that our organization still uses > the older visualstudio domain format
> `https://org.visualstudio.com` instead of `https://dev.azure.com/org`
Do you think that this could be a common problem? Any ideas how SmartGit could detect this problem?
the older domain is always visualstudio.com with the format: org.visualstudio.com so yes, it could definitely be detected.
At which point would SmartGit get in touch with an "org.visualstudio.com"-URL? None of your above logs mentions "org.visualstudio.com".
Hi Marc, I'm not sure I totally understand the question, but from a user perspective, once they add a Hosting Provider of type Azure DevOps, then any repo the domain dev.azure.com or *.visualstudio.com should be treated as Azure DevOps repo.
One has the Organization name in the domain, the other in the path, that's the only difference.
Update:
Regenerated the Token and am now seeing the PRs :)
:D That is really good to hear! Hence, from my understanding following issues are still open:
- some error messages are still mentioning "Bitbucket"
- the rundll32-link should better be encoded: it seems encoded version is working for all of us with all browser
- improve "no refresh-token present" error message
- personal access tokens: I can reproduce the 401 for "https://app.vssps.visualstudio.com/_apis/profile/profiles/me" now -- I guess since introducing that call I didn't check again with PATs, sorry for that ...
Just to be complete,
- The ability to choose the organization should be added otherwise it is impossible to use an organization that is not the first entry.
- `Account` should be renamed to `Organization` to be clearer as one can get it confused with email account.
- Lastly, for PAT to work, one needs to provide an email address, so that field needs to be requested in the PAT dialog along with the Organization.
I'm wondering whether we need accounts/usernames at all. OAuth tokens/PATs already seem to uniquely identify the users. I couldn't find any official statement in the REST API docs about this, but I found the following:
https://github.com/MicrosoftDocs/azure-devops-docs/issues/2455#issuecomment-439503194
The same is also true for GitHub, Bitbucket and GitLab. For my experiments, I could now also successfully use the OAuth token and a PAT to access repositories of my two test organizations. For PATs, it's important to give them access to "All accessible organizations"; scope Code Read&Write should be sufficient.
@hrant and @Martillo, can you please confirm that you are able to access all of your (private) repositories when creating such a PAT:
Note that there is another bug in SmartGit's latest build which prevents PATs from working correctly (almost always), hence it's not yet expected to work from within SmartGit.
I would expect this to work
but I don't like including the username/PAT in the repo properties, I much rather add a hosting provider which would hide the details (for security), and let SmartGit use the credentials from there.
Does that make sense?
And for that to work, when adding a hosting provider, I would need to provide username and PAT
PS: come to think of it, were you literally asking if `anything` instead of the username would work with PAT.
I have an Azure DevOps project that I created a while back to test branching in Azure DevOps. In SmartGit, the origin for the repos in this project in SmartGit was still set to the old https://[org].visualstudio.com URL, although a while back I had switched to the new https://dev.azure.com/[org] format. Nevertheless, I was still able to use SmartGit to push changes to the remote branch, but the Azure logo wasn't being displayed in the SmartGit "Branches" window. When I changed the origin in SmartGit to point to https://[org].visualstudio.com then the Azure logo showed up. However, there's an active pull request on this repo, but I don't see it in SmartGit, even after regenerating the token and selecting View > Refresh in the menu.
One note about creating the token. I have two Azure DevOps orgs configured as hosting providers in SmartGit (call them Org A and Org B). If I open up the Hosting Providers dialog and:
@Martillo
> although a while back I had switched to the new https://dev.azure.com/[org]
format. Nevertheless, I was still able to use SmartGit to push changes
to the remote branch,
Both formats works for now, but according to MS notes, see this
https://docs.microsoft.com/en-us/azure/devops/release-notes/2018/sep-10-azure-devops-launch#administration
> but the Azure logo wasn't being displayed in the
SmartGit "Branches" window. When I changed the origin in SmartGit to
point to https://[org].visualstudio.com then the Azure logo showed up
Interesting, for me it was the inverse, we're still using the older domain for the concern highlighted above, and only after changing the repo properties in Smartgit to point to dev.azure.com I was able to get the Azure Logo
You can manually change the Organization ("Account") in the Azure DevOps account dialog. But as pointed out in my last comment, I hope to get rid of the Account entirely. Can you please give the clone, as outlined above, a try?
@Marc,
As I previously reported, manually changing the Account never worked for me, I couldn't add the second Organization as a Hosting Provider, even as a first Azure DevOps hosting Organization.
@Marc - apologies, I didn't see your earlier post last night when I added the post above.
I just ran the code below at the command line and successfully cloned two repos from projects in two different orgs that are both accessible via a single PAT that's tied to one Azure DevOps profile (which appears to be tied to a specific Microsoft account). I was also able to clone another repo in another org using a different PAT I created in another Azure DevOps account that's tied to a different Azure DevOps profile/MS account. No dialogs popped up to ask me to log into my MS account, so it was all very smooth.
I can confirm that anything (not username) worked for me as well.
I learned something new today, so PATs don't need the associated username.
Thanks
Since all you need is a PAT to give a user access to one or more of the Azure DevOps organizations they have access to via a given Microsoft (or Azure AD) Account, and since a user may have access to multiple Azure DevOps instances using different Microsoft/Azure AD logins, I'm thinking that what the Hosting Providers configuration in SmartGit should provide is a way to record and track the different Azure DevOps PATs a user has, and which Azure DevOps organizations each PAT gives them access to so that SmartGit can inject the correct PAT into a request to connect to a given Azure DevOps repo.
I'm not planning to support PATs which are limited to certain organizations. Note that PATs are anyway only a workaround in case your have troubles with OAuth authentication (usually you shouldn't have). Such limited PATs would make several code parts more complicated. For example, already "_apis/profile/profiles/me" fails.
If you have different user accounts (=emails), that should be supported by setting up one Azure DevOps Hosting Provider per account.
@hrant and @Martillo, thanks for your testing and feedback! I have now uploaded build 17071 (Help|Check for Latest Build) with following changes:
Note, that:
@Marc - thanks for working on this! I'm going to do some testing as soon as I can carve out a few free minutes
I played around with the new build for a little bit and it looks good! The announcement of the new Azure DevOps feature in the lower right corner at startup is awesome.
I tested connecting using an OAuth token when "Allow Third-party Application access via OAuth" was turned off for the org in Azure DevOps (see screenshot) and got an HTTP 401 response, so it would helpful to add some instructional text in the dialog box to note that this needs to be turned on in the org. Also, I had real problems creating an OAuth token for an org where I authenticate with a different MS account than the one I'm logged into my Windows workstation with, so I gave up and used a PAT instead. Since authenticating with OAuth tokens for multiple MS accounts could be problematic, you might get fewer support request from users if you encourage PATs over OAuth. For PATs, it would also be helpful to add some text on the Azure DevOps Configure Hosting Provider dialog to explain that a PAT only needs Code Read/Write permissions but access to all accessible orgs.
One note: I think it would be nice if SmartGit indicated which authentication method is being used - PAT or OAuth - when you edit an existing Azure DevOps hosting provider.
The Pull Requests link is showing up but I haven't experimented much yet with the functionality. BTW , by "Retract Pull Request" do you mean "Abandon"? I think the official Azure DevOps term for this is "Abandon Pull Request" (https://docs.microsoft.com/en-us/azure/devops/repos/git/pull-requests?view=azure-devops)
> I tested connecting using an OAuth token when "Allow Third-party Application access via OAuth" was turned off for the org in Azure DevOps (see screenshot) and got an HTTP 401 response, so it would helpful to add some instructional text in the dialog box to note that this needs to be turned on in the org.
Thanks for this hint! Usually we collect possible problems with Hosting Provider setups in our docs, hence I have added this information there: https://www.syntevo.com/doc/display/SG/Azure+DevOps
> Also, I had real problems creating an OAuth token for an org where I authenticate with a different MS account than the one I'm logged into my Windows workstation with, so I gave up and used a PAT instead.
So you can't simply switch accounts in your browser? This works fine for e.g. GitHub/Bitbucket, but I understand that for Azure authentication maybe somehow be tied to your workstation login. In this case, I agree, PATs may be the best choice.
> Since authenticating with OAuth tokens for multiple MS accounts could be problematic, you might get fewer support request from users if you encourage PATs over OAuth.
I'm wondering how common it is with Azure that one user has multiple accounts? Why do you as a single person have multiple accounts? Note that you wrote about "multiple Azure DevOps instances" which is something different: there must be a way to switch between them in the browser and then OAuth should work out-of-the-box.
> For PATs, it would also be helpful to add some text on the Azure DevOps Configure Hosting Provider dialog to explain that a PAT only needs Code Read/Write permissions but access to all accessible orgs.
I have checked related code, but it can only distinguish between automatic generated tokens and manually entered tokens. After the token has been confirmed such an info could be displayed. But especially during setup while the user may type/paste anything into the Token text field, I can't tell whether this is a PAT or not. Hence, providing a reliable text here is a bit tricky.
> BTW , by "Retract Pull Request" do you mean "Abandon"?
SmartGit is using "Retract" for outgoing pull requests. For all Hosting Providers, the underlying commands which will be executed are identical to Rejecting a pull request. Hence, for the next build, I have switched from "Retract" to "Reject", except for Azure DevOps which will use "Abandon" instead of "Retract"/"Reject".
> So you can't simply switch accounts in your browser? This works fine for
e.g. GitHub/Bitbucket, but I understand that for Azure authentication
maybe somehow be tied to your workstation login. In this case, I agree,
PATs may be the best choice.
In our case, we can switch accounts in the browser as long as we sign out or open an incognito / private browser session and sign in there.
> I'm wondering how common it is with Azure that one user has multiple accounts?
I can't tell how common it is, but we also have / use multiple accounts.
I can imagine developers working from home having their personal accounts as well as work accounts would be common. Contractors possibly could be using multiple accounts as well.
> SmartGit is using "Retract" for outgoing pull requests. For all Hosting
Providers, the underlying commands which will be executed are
identical to Rejecting a pull request. Hence, for the next build, I have
switched from "Retract" to "Reject", except for Azure DevOps which will
use "Abandon" instead of "Retract"/"Reject".
Azure DevOps also has Reject for Pull Request, which is different than Abandon,
You Abandon a PR if you no longer want that PR to be merged.
You Reject (as a reviewer) a PR because it requires changes before it is acceptable to be merged.
Not sure which is equivalent to the retract.
Hi Marc,
Thanks for continued work on this.
I did a quick test,
Upgraded to the latest
deleted the Azure Devops hosting provider
Used OAuth, entered the Authorization code, and smartgit got stuck at this
It looks like it got past org1 but got stuck on org2
This is what I see in the logs
7m24s510 (2021-04-09 15:36:43) [main] INFO q.swt.shellActivation - Activation change from Shell {Add Hosting Provider}@3ED7DD70 to Shell {Checking connectivity}@5F935D497m24s512 (2021-04-09 15:36:43) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO smartgit.azure - Sending OAuth request https://app.vssps.visualstudio.com/oauth2/token
7m24s524 (2021-04-09 15:36:43) [main] INFO q.swt.shellActivation - Activation remains at Shell {Checking connectivity}@5F935D49
7m24s726 (2021-04-09 15:36:43) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO q.json - Sending GET 'https://app.vssps.visualstudio.com/_apis/profile/profiles/me'
7m24s871 (2021-04-09 15:36:43) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO q.json - Sending GET 'https://app.vssps.visualstudio.com/_apis/accounts'
7m25s 31 (2021-04-09 15:36:43) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO q.json - Sending GET 'https://app.vssps.visualstudio.com/_apis/accounts'
7m25s285 (2021-04-09 15:36:44) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO smartgit.azure - Sending OAuth request https://app.vssps.visualstudio.com/oauth2/token
7m25s489 (2021-04-09 15:36:44) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO q.json - Sending GET 'https://dev.azure.com/org1/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=0'
7m25s706 (2021-04-09 15:36:44) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO q.json - Sending GET 'https://dev.azure.com/org2/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=0'
7m25s921 (2021-04-09 15:36:44) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO q.json - Sending GET 'https://dev.azure.com/org2/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=50'
7m26s369 (2021-04-09 15:36:45) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO q.json - Sending GET 'https://dev.azure.com/org2/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=100'
7m26s686 (2021-04-09 15:36:45) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO q.json - Sending GET 'https://dev.azure.com/org2/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=150'
7m27s138 (2021-04-09 15:36:46) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO q.json - Sending GET 'https://dev.azure.com/org2/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=200'
7m27s540 (2021-04-09 15:36:46) [ThreadPool-0 (smartgit.TM$$Lambda$853/0x000000010070d040)] INFO q.json - Sending GET 'https://dev.azure.com/org2/_apis/git/repositories?api-version=6.1-preview.1&$top=50&$skip=250'
For my small test organization the paging logic works. Is it possible that your organization has really many repositories and SmartGit is not actually stuck but it only takes a long time to complete? When comparing:
and:
in your browser, are results different?
Yes I do get different results, they are two different Organizations,
The first one has a count of 7, the second one which hangs has a count of 119
The loading in a browser is almost instantaneous for both.
Is Smartgit processing each entry?
Thanks, I understand now. Paging ($top,$skip) isn't supported for that API. I have uploaded build 17075 which should fix this problem. This most likely also explains the out-of-memory error: SmartGit reads and caches an infinite amount of JSON objects and finally runs out-of-memory.
When exactly did this happen?
> SmartGit is not actually stuck but it only takes a long time to complete?
I had forgotten to kill it when it was hung on adding the hosting provider, it had eventually ran out of memory.
Marc,
I know you mentioned that visualstudio.com domain is not supported, I'm not sure if this is for now or there are no plans to support it, either way I think the documentation can benefit from mentioning this.
I my humble opinion, it shouldn't be much effort to support both, as the difference is only in the base URI.
re: supporting Azure DevOps orgs that use the visualstudio.com URL: These still work in SmartGit, but you don't get the added Azure DevOps specific features. The visualstudio.com URL is deprecated, but I agree that full Azure DevOps support for it would be nice since it can be time-consuming for dev teams to change the origin URL for a bunch of active repos. However I would rate support for this a lower priority and schedule it for a future release if you don't have bandwidth to work on it now. I agree there should be something in the docs about it.
re: generating an OAuth token for org #2: the other day, I copied the URL that generates the token, then logged into the Azure DevOps account for org #2 using my separate Org #2 Microsoft account, pasted the URL into the browser location bar, hit enter and if I recall correctly a token was generated but when I tried to use it, it didn't work. I tried about three times, but didn't have time to pursue it any further than that.
re: one user > multiple Azure DevOps accounts. I think this could be very common. I have an account for my consulting business. I also had an account at my previous day job (where I purchased 20 or 30 or maybe more SmartGit licenses), and another account with a separate MS login that belongs to a company for which I'm doing some consulting.
re: viewing a hosting provider's authentication method (PAT or OAuth) when managing hosting providers: nice to have but I'm sure users can live without it.
re: "Abandon" and "Reject" : I agree with hrant that both actions should be available for Azure DevOps repos. To me the words "Abandon" and "Retract" express the same concept.... There is a StackOverflow post from August 31 2020 stating "GitHub now supports closing a pull request" - GitHub Close looks like it corresponds to the Azure DevOps Abandon action.
@hrant and @Martillo, again thanks for your testing and feedback. I have now uploaded build 17074 which comes with following changes/fixes:
@Marc
Thank you for the update.
Quick feedback.
Adding the hosting provider finally worked, and it recognized both organizations, furthermore visualstudio.com domains also worked.
This is based on quick check, I'll report back once I start using it heavily.
Thanks for all your efforts.
@Marc
Sorry, I mistakenly thought that visualstudio.com domain was working fine while I was looking at dev.azure.com domain.
Here's a scenario that is not working, but am sure it can easily be handled.
For arguments sake, let's say that we have the following variables
organization
project
repository
The following format is valid in visualstudio.com domain
organization.visualstudio.com/DefaultCollection/_git/repository
Notice how the project is DefaultCollection (blame MS for that, for projects that had only single repository)
The equivalent of the above would be:
organization@dev.azure.com/organziation/repository/_git/repository
Notice how project DefaultCollection took the name from the repository.
Hope this help.
Thanks! Build 17078 should now correctly handle "DefaultCollection".
Thank you Marc,
Can confirm that Build 17078 is handling "DefaultCollection" properly.
Thanks for a quick resolution.
@Marc - thanks for all the hard work! I grabbed the latest version and was able to successfully click through from SmartGit to view a pull request in a web browser, as well as successfully abandon a pull request from within SmartGit. Looks pretty good! The only remaining issue, if it is one, is that repos where the origin URL is specified as https://[org].visualstudio.com don't show the Azure logo in SmartGit.
> repos where the origin URL is specified as https://[org].visualstudio.com don't show the Azure logo in SmartGit.
That's unexpected. Such URLs should work since build 17074 and "DefaultCollection" should be handled correctly since build 17078. @hrant has confirmed this -- please see our discussion above. Are you referring to exactly this kind of URLs or am I misunderstanding?
Indeed @Martillo, it is working for us
The notification was not yet triggered for "visualstudio.com" domains. This is fixed now in build 17079.
Hi Marc, I want to thank you for driving this integration, this is going to help a lot.
So far I have not encountered any issues and am very happy with it.
If I may, let me share what else we do with our own in-house integration that we would miss if we used this solely.
Perhaps there is an opportunity for Smartgit to add that as well as be a true differentiator from other git solutions.
I think it is possible considering that SmartGit already has a Jira Integration.
What we basically do is parse the commit message, and detect a Jira Ticket pattern (Smartgit Already does this), if we find any, we do the following
1- In Azure DevOps PR Ticket, we add a Description to the Overview of the Pull Request, something like this, linking back to Jira ticket.
2- We add a comment into the Jira Ticket, linking back to the Pull Request
Something like this
3- We transition the Jira ticket to In Review however I don't think this one is necessary for Smartgit to handle as it would complicate things.
Do you think 1 and 2 are feasible?
Many thanks
Are you doing this on the client-side? To me this sounds more like a server-side task which would be triggered by some post-receive hook. Are all three steps done at the same time? If so, when exactly?
Yes, all on the client side, when a Pull Request is created.
OK. This sounds like a very specific task designed for your environment. I'm still not sure that the client-side is the appropriate place for it because you have to ensure that everyone is using a client which performs these additional tasks. Azure Webhooks don't seem to be useful here, but Pipelines might be. Also, there already seem to be some existing integrations between Azure and JIRA already [1].
Even on the client side, this can be implemented outside of SmartGit, e.g. by Git client-side hooks. Hence, I don't think we would address this issue in SmartGit.
[1] https://devblogs.microsoft.com/devops/azure-pipelines-integration-with-jira-software/
Fair point Marc, thanks for your feedback.
I'll look into the other options you mentioned.
Thanks again.
In case others want to try https://devblogs.microsoft.com/devops/azure-pipelines-integration-with-jira-software/
It didn't even install for us. :(
@Marc and @Hrant - my visualstudio.com test above may not be legit...
My org is currently configured to use the new https://dev.azure.com/[org name] URL, but I can still pull and push code from SmartGit using the old https://[org name].visualstudio.com/_git/[repo name] URL. I discovered this because there were some repos that I hadn't worked with in a while which were still configured in SmartGit to use the old URL scheme. Pull and push works, but the Azure icon doesn't display and no Pull Request information is available. There's no "DefaultCollection" element in this URL; I don't know if my Azure DevOps repo URLs ever included a "DefaultCollection"...
I can confirm that the URL of your origin is not yet supported: it's missing the "project" part. My visualstudio.com URLs look like: https://org.visualstudio.com/project/_git/repo
Now I'm wondering why this "project" part is missing for you? It seems that I can only create repositories within a project...
@Martillo,
Can you try the following?
Change your visualstudio url to include the project, like Marc is suggesting, and use the Repository name as project name
It should be something like this for your example:
https://[org].visualstudio.com/Utilities/_git/Utilities
Or better yet, open a browser to [org].visualstudio.com
Navigate to that Utilities repo, and copy the clone link, see what it offers.
Here's what I suspect might be happening.
That URL is working for you, because in that project there is a single repository and it would probably also work with these two URLS
https://[org].visualstudio.com/Utilities/_git/Utilities
https://[org].visualstudio.com/DefaultCollection/_git/Utilities
> Now I'm wondering why this "project" part is missing for you? It seems that I can only create repositories within a project.
That's correct, hence why there is always the DefaultCollection, even if it is a single repo,
If I recall vaguely they supported access to the repo even if DefaultCollection project part was not in the URL.
Hi Marc,
Not sure if there are future plans to add some of the missing Actions, as that would keep one's work totally in SmartGit and eliminate the need to jump to the browser to take those actions.
Don't get me wrong, what you've done is already great, just thinking how it could be improved upon.
Thanks
Approve definitely makes sense. There are similar topics for GitHub and Bitbucket, too. As always, we will probably not support every possible option but try to find some common GUI which will work for all Hosting Providers. I guess that would be Approve vs. Reject with an optional comment. How important is the auto-complete functionality?
Please post a new UserEcho topic for "Azure DevOps: Approve Pull Requests" and include information on must-have and nice-to-have options.
Thanks I will,
Auto-complete is very useful, specially if you have other pipelines pending on the task completion,
Once all requirements are met,
ie
- all commented issues are marked resolved
- Reviewer(s) approved
- Sonarqube / Lint passes
- Build passes
The merge would be completed without user action and deployment pipeline triggered.
I will post into UserEcho, thanks
Hi Marc,
Couple of observations
Not sure when an how SmartGit syncs the comments from a Pull Request
1- The only way I could make Smartgit reflect the changes done on the Web is by toggling / un-toggling , not sure if that is by design
2- Actions on Comments are missing, but I'm sure you know about that
3- If I make a code inline comment in SmartGit, it is not reflected on the Web, the reverse works, but I need to do the toggle dance to see the changes.
Thanks.
> 1- The only way I could make Smartgit reflect the changes done on the Web is by toggling / un-toggling , not sure if that is by design
Clicking the Azure toolbar button should refresh all information. Refreshing the Log view should also refresh all information. After any Azure-related operation information should automatically be refreshed.
> 2- Actions on Comments are missing, but I'm sure you know about that
In the Window|Comments view there should be Edit, Reply, Delete. In the Changes view, if a pull request node is selected in the Log, it should also be possible to add new comments.
> 3- If I make a code inline comment in SmartGit, it is not reflected on the Web, the reverse works, but I need to do the toggle dance to see the changes.
I can't reproduce that. I guess it will be time for a shared test repository. Will it be possible for you to set this up?
Thanks Mark
1- Clicking Azure Toolbar button does actually refresh, thanks
2- Thanks, saw those, I meant actions like these
3- Overall comment syncs, code inline comment sync does not (only from SmartGit to Web), if that is working for you, I'll look into setting up a shared repo.
Re: the URL issue - "Utilities" is the name of the project. "Utilities" is also the name of the only repository in that project. Ultimately it's not an issue since I can simply change the origin in SmartGit to use the dev.azure.com URL, but if I get some time later today I'll try your suggestions.
Build 17083 now supports this optimized URLs, for "visualstudio.com" as well as "dev.azure.com" domains. On the other hand, I have added another check to ensure that the remote URL ends with the repository name and does not contain additional, unexpected path segments. Hopefully that doesn't break URLs which have already been working now :)
Bingo! Thanks Marc!
@Marc
I don't know if this is related to Azure DevOps changes or not, I have this annoying issue that every time I bring Smartgit to foreground I'd have to wait 20-30 seconds for it to be active, it's probably refreshing some stuff but it cannot be used as it grays out the buttons, it's very troubling.
Thanks
Azure Refreshing shouldn't block the GUI and repository refreshing for 30 seconds sounds excessive. Please send periodical thread dumps to smartgit@syntevo.com:
https://www.syntevo.com/doc/display/SG/Periodical+Thread+Dumps
Hi,
I tried to add a Azure DevOps hosting provider with our custom (on premise) DevOps server, using a token generated there. But upon pressing "Add" it fails.
I see it tries to access visualstudio.com, but since I chose the custon server, this seems a bit odd.
Any idea's on this?
Running 21.1 preview 6 #17089
Thanks for your feedback! You are right. SmartGit wasn't actually prepared for on-premise instances. I have now uploaded build 17090 which should work better: Help|Check for Latest Build
I was able to confirm that the REST API works for my test setup (VM). I wasn't able to access repositories using Git from outside of the VM: I'm always ending up with "Authentication failed for ...", no matter which username/PAT combinations I'm using. How do you access your Git repositories from command line?
Another note: for every REST call, the "api-version=" is included, which currently defaults to "6.0". This seems to work with the most recent Azure DevOps Server. If you encounter error messages related to the API version, you can change this in the Preferences, Low-Level Properties, "azure.apiVersion".
Thanks, I can add the hosting provider now!
And thanks for pointing to the api-version-setting, needed to change that to 5.1
When I clone a repo, I can see and select all repo's on our Azure DevOps server, and cloning works. However, I do not get the Azure toolbar. It should be just there in the Branches-panel? Anything I can look for in the logs?
> How do you access your Git repositories from command line?
I'd say just normal. The git credential manager does its job. I vaguely remember having "Authentication failed for..." issues a few years back, but can't really remember the fix for that, sorry.
Yes, the button should be there.
Please enable TRACE logging for "smartgit.azure.config". For details, have a look at: https://www.syntevo.com/doc/display/SG/Debugging#Debugging-Enablingdebugloggingforcertainkeys
Once enabled, logs/logs.txt.0 will contain details on where the mapping between your Hosting Provider configuration and the remote URL fails.
Your screenshot shows the URL ending with "/tfs". This sounds like the root of all collections, though for me that would simply be "http://server:8080/". The REST API always requires to provide a "collection", hence I'm using "http://server:8080/DefaultCollection" as Server URL. Do you repository URLs also contain this kind of "collections"? Do you have several different "collections"? Some example URLs would be helpful.
This seems to be the relevant part of the log (replaced actual server with 'myserver'):
4s291 (2021-04-29 10:41:05) [WorkerThread-1] FINE smartgit.azure.config - Creating configuration ...
4s294 (2021-04-29 10:41:05) [WorkerThread-1] FINE smartgit.azure.config - for http://myserver:8080/TFS/OFCollection/REM/_git/RegTestRunner
4s295 (2021-04-29 10:41:05) [WorkerThread-1] FINE smartgit.azure.config - ... url = http://myserver:8080/TFS/OFCollection/REM/_git/RegTestRunner
4s295 (2021-04-29 10:41:05) [WorkerThread-1] FINE smartgit.azure.config - ... urlHost = myserver:8080
4s295 (2021-04-29 10:41:05) [WorkerThread-1] FINE smartgit.azure.config - ... '_git' not found
4s295 (2021-04-29 10:41:05) [WorkerThread-1] FINE smartgit.azure.config - ... null
The screenshot was before last build, where I tried different things ("8080/", "8080/tfs", "8080/tfs/OFCollection"). Currently the Server URL ends with "/tfs/OFCollection", which is the only collection. REM is a project within that collection (there exist several projects) which in turn contains multiple git repo's.
I tried changing the Server URL to contain the project (so "...:8080/tfs/OFCollection/REM"), but got the same logs as above.
This did change one thing: the list with available projects (when cloning) showed only REM-repos, whereas before it showed repos from all projects.
The corresponding lines from the log (/tfs/OFCollection vs /tfs/OFCollection/REM):
2m57s142 (2021-04-29 00:19:29) [ThreadPool-3 (smartgit.TM$$Lambda$909/0x000000010073c440)] INFO q.json - Sending GET 'http://myserver:8080/tfs/OFCollection/_apis/git/repositories?api-version=5.1'
12m 7s613 (2021-04-29 10:53:08) [ThreadPool-7 (smartgit.TM$$Lambda$964/0x00000001007ccc40)] INFO q.json - Sending GET 'http://myserver:8080/tfs/OFCollection/REM/_apis/git/repositories?api-version=5.1'
Thanks, please give build 17091 a try which should now be able to handle URLs with some prefix like "tfs/". Your server URL should be "http://myserver:8080/TFS/OFCollection".
It works, thanks!
Just added my first PR comment from SmartGit, but Azure DevOps has some trouble with it.
The top comment is from SmartGit, the bottom comment is added from Azure DevOps itself:
Difference seems to be the missing starting "/".
If I click the "View the comment in its original context"-link, it still cannot find the file. If I then change the URL from "...?path=src%2FTextReporter.cpp&..." to "...?path=%2Fsrc%2FTextReporter.cpp&..." it at least finds the file (but without the comment, since that was added to the 'non existent' file without the "/")
Thanks! I have now uploaded build 17092 which fixes this issue and another possible problem with replies linking to deleted parent comments.