bMjUdZddlZddlZddlZddlZddlZddlZddlmZddl Z ddl Z ejdZ ee jjdz ZdefdZdefdZej*j-d d Zej*j-d d Zej*j-d d Zerd end Zej*j-dd Zej*j-ddZej*j-ddZdZdgZdZ dZ!dZ"dZ#iZ$eee%e&effe'd<defdZ(dpdededzdedzfdZ)dedzfdZ*dedefdZ+dede,edzfd Z-ded!ed"ede.dzfd#Z/ded$ed"ede.dzfd%Z0ded$ed&ede,edzfd'Z1 dqded$ed&ed)e2d*edzd+edzdefd,Z3 drded$ed"ed-e.d.ededzf d/Z4ded$ed0ede,edzfd1Z5ded$ede,edzfd2Z6ded$ed0ededzfd3Z7ded$ed0ede.dzfd4Z8ded$ed&ed5ededzf d6Z9dd(ddd7ded&ededzd)e2d*edzd+edzdefd8Z:dd9ded&ed5ededzdedzf d:Z;dd9ded"ededzde.dzfd;Zddddd>ded?ed@edAe,edBedzdCe,edzdDe,edzdEe,edzdedzfdFZ?dddGded?ed@edAe,edBedzdDe,edzde@fdHZAddIdJdedBedzd)e2de,edzfdKZBddLdedMedNedBedzde%e.eefdzf dOZCddLdedMedBedzde@fdPZDdQZEdQZFdedRedSe.ddTfdUZGdddVdedWe.dXedzdYedzde.dzf dZZHded!ed0ede@fd[ZIdsd\ed-e.d.edefd]ZJdtd_edzd`ede@fdaZKeLdbk(rddlMZMddlNZNeMjdcdZPePjdedfdghePjdidjkePjdld^dmnePjZSeSjr7eNjeKeSjeSjrdydoyePjyy)uu8 winston_graph.py — App-only Microsoft Graph client for Winston (Email Bot). Provides file-read (OneDrive/Files.ReadWrite.All) and mail-draft-create (Mail.ReadWrite scoped to sales@icastinc.com) operations using the same MSAL client-credentials pattern as notifications.py. Required env vars (same as Togen's existing Graph setup): AZURE_CLIENT_ID App registration client/application ID AZURE_CLIENT_SECRET App registration client secret AZURE_TENANT_ID Azure AD tenant ID Winston-specific env vars: WINSTON_ONEDRIVE_USER UPN or object ID of the user whose OneDrive holds the Takeoffs/Completed Quotes folders (e.g. "administrator@icastinc.com") WINSTON_MAIL_FROM Mailbox to create drafts in (default: sales@icastinc.com) N)Pathztogen.winston_graphzwinston_settings.jsonreturnctjr= tjtj }t |t r|S iSiS#tjtf$rYiSwxYw)zLoad winston_settings.json (best-effort). Saved values take precedence over env vars for Azure/OneDrive config so admins can update them in the UI without editing the systemd service file (DVI-829).) _SETTINGS_FILEis_filejsonloads read_text isinstancedictJSONDecodeErrorOSError)datas $/var/www/html/togen/winston_graph.py_winston_settingsr%so ::n6689D$% & I2I$$g.   I s8AA10A1cFtjddxstS)zGEffective OneDrive user UPN: settings file preferred, env var fallback. onedrive_user)rgetWINSTON_ONEDRIVE_USERrget_onedrive_userr3s   " "?B 7 P;PPrAZURE_CLIENT_IDrAZURE_CLIENT_SECRETAZURE_TENANT_ID"https://login.microsoftonline.com/rWINSTON_MAIL_FROMzsales@icastinc.comWINSTON_SERVICE_USERztogen@icastinc.comzwinston-pipeline-scratchz$https://graph.microsoft.com/.defaultz https://graph.microsoft.com/v1.0z7icastinc.sharepoint.com:/sites/InfrastructurePrecastIncz[sharepoint-site]g @_site_drive_cachecFtjddxstS)zAEffective SharePoint site (settings preferred, built-in default).sharepoint_siter)rr_DEFAULT_SP_SITErrrget_sharepoint_siter$Vs   " "#4b 9 M=MMrtokensitec|xs t}tj}tj|}|r||dz tkr|dS t jt d|t|d}|jdk7r0tjd|j||jdd y|jjd }|stjd |yt jt d|d t|d}|jdk7r/tjd |j|jdd y|jjd }|stjd|y||ft|<|S#t$rtjdYywxYw)a-Resolve the default document-library driveId for a SharePoint site. ``site`` is the "{host}:/sites/{name}" server-relative form; defaults to the InfrastructurePrecastInc site (DVI-896/DVI-947). Cached for ``_SITE_DRIVE_TTL`` seconds to avoid a two-call resolve on every pipeline run. rz/sites/headerstimeoutz9resolve_site_drive_id: site lookup failed (%s) for %s: %sN,idz-resolve_site_drive_id: site %s returned no idz/drivez3resolve_site_drive_id: drive lookup failed (%s): %sz6resolve_site_drive_id: site %s default drive has no idzresolve_site_drive_id error)r$timer r_SITE_DRIVE_TTLrequests _GRAPH_BASE _auth_header status_codelogerrortextr Exception exception)r%r&nowcachedrsite_iddrdrive_ids rresolve_site_drive_idrA[s  (&(D ))+C  " "4 (F 3?o5ay LLK=v6 U@S]_ ` ==C  IIQmmT166$3< 9&&(,,t$ IIEt L \\[M @".u"5rC >>S IIKnnbggdsm 5779==& IINPT U#&/$  34s,A(F87F0A(F7F FF?>F?ct}|jddxst}|jddxst}|jddxst}|rd|nd}|r|r|st j dy tj|||}|jt }|jd }|s&t jd |jd ||S#t$rt jd YywxYw)zAcquire an app-only Graph token via client credentials. Returns token or None. Saved settings take precedence over env vars (DVI-829).azure_client_idrazure_client_secretazure_tenant_idrzHGraph token unavailable: AZURE_CLIENT_ID/SECRET/TENANT_ID not configuredN) authorityclient_credential)scopes access_tokenzGraph token request failed: %serror_descriptionzGraph token acquisition error)rr _AZ_CLIENT_ID _AZ_SECRET _AZ_TENANTr6warningmsalConfidentialClientApplicationacquire_token_for_client _GRAPH_SCOPESr7r9r:)s client_idsecrettenantrFclientresultr%s racquire_graph_tokenrYs A',= I+R0>JF',: FAG4VH=RI &Y ^_ 33 fF00 0F >* II6 CVX^8_ `  56s?A(C((D D cd|ddS)NBearer application/json Authorizationz Content-Typer)r%s rr4r4s&ug.@R SSrcX tjtdt|d}|jdk7r/t j d|j|jddy|jjdgS#t$rt jd YywxYw) zBList all drives visible to the app (useful for initial discovery).z/drivesr)r*r-zlist_drives failed (%s): %sNr.valuezlist_drives error) r2rr3r4r5r6r7r8rr9r:)r%resps r list_drivesrbs||{m73\%=PZ\]   s " II3T5E5EtyyQURU Wyy{w++  )*sA%B(BB)(B)r@ drive_pathc:td|d|d} tj|dd|id}|jdk7r0tj d |j||j d d y |jS#t$rtjd Yy wxYw) zDownload raw bytes of a file at drive_path inside the specified drive. drive_path is relative to root, e.g. "Takeoffs/2026-06-29_tasks.json". Returns file bytes or None on error. /drives//root:/ :/contentr^r[<r*r-z&drive_read_file failed (%s) for %s: %sNr.zdrive_read_file error r3r2rr5r6r7r8contentr9r:)r%r@rcurlras rdrive_read_filerls M(7:,i HC ||C/WUG;L)MWYZ   s " II>&& DIIdsO E||  -.sAA9- A99BBuser_upnc <td|d|d} tj|dd|id}|jdk7r1tj d |j|||j d d y |jS#t$rtjd Yy wxYw) zDownload a file from a user's OneDrive by UPN. Requires Files.ReadWrite.All (application). drive_path is relative to root. /users/ /drive/root:/rgr^r[rhr*r-z.user_drive_read_file failed (%s) for %s/%s: %sNr.zuser_drive_read_file errorri)r%rmrcrkras ruser_drive_read_filerqs M - |9 MC ||C/WUG;L)MWYZ   s " IIF&&*diio O||  23sAA:. A::BB folder_pathc ltd|d|d} tj|t|d}|jdk7r1t j d|j|||jdd y|jjd gS#t$rt jd YywxYw) zGList folder contents from a user's OneDrive. Returns item list or None.rorp :/childrenr)r*r-z0user_drive_list_folder failed (%s) for %s/%s: %sNr.r`zuser_drive_list_folder error r3r2rr4r5r6r7r8rr9r:)r%rmrrrkras ruser_drive_list_folderrvs M - }J OC ||Ce) ?C ||Ce)2 A>>BB quote_numberctd|d|d}ddd}d}|r tj|t|d|vr|nd d }d }|jd k7r/t j d |j|jd dy |j}|jdgD]X} | jdd} |dz }|| vs| jjds?t jd| || cS|jd}|rt jd|||y #t$rt jdYy wxYw)zFind the xlsm workbook whose filename contains quote_number in a OneDrive folder. Scans up to ~4,700 .xlsm files. Returns the first matching item dict or None. Supports paged results ($top=500) to handle large folders efficiently. rorprtrUid,name,size,lastModifiedDateTime,webUrl,parentReference,@microsoft.graph.downloadUrlrr?Nrhrr-z0find_quote_workbook folder fetch failed (%s): %sr.r`namerr(.xlsmz5find_quote_workbook: found %s after scanning %d itemsrzfind_quote_workbook errorz@find_quote_workbook: quote %s not found in %s (scanned %d items))r3r2rr4r5r6r7r8rlowerendswithinfor9r:rN) r%rmrrrrkrseenraritemrs rfind_quote_workbookrusZ M - }J OC&} ~F D  <<\%-@SV^aSagkuwxDF3& LdN^N^`d`i`ijnkn`op99;D"- xx+ 4'DJJL,A,A',JHHTVZ\`aK  ((,-C $KKRT`bmost    MM5 6 s+A'D,AD,D,#D,>D,,E  E )r&rxryrzc |r"|jtsddddddS|d}}nAt||}|sddddd|xs tddStd|d |d }|d d }|r||d < t j |t ||d} | jdk7rp|r |s| jdk(rt||||Stjd| j|| jdd| jddd| jdddS| j} | j d} d| j dgt!| | ddS#t$r0} tjdddddt| dcYd} ~ Sd} ~ wwxYw)u Diagnostic folder listing on the SharePoint site drive (mirror of ``user_drive_list_folder_detail`` for the shared Files views — DVI-947). Supports ``order_by`` ($orderby) + ``next_link`` (paging cursor) for the sort/Show-more Files controls (DVI-978).rNFr|r}z)could not resolve SharePoint site drive ()rerfrtrrrr)rz#site_drive_list_folder_detail errorr-r)r&rxz4site_drive_list_folder_detail failed (%s) for %s: %sr.rrr`)rr3rAr$r2rr4r9r6r:rr5site_drive_list_folder_detailr7r8rr) r%rrr&rxryrzrkrr@rarrrs rrrs##K0$U!%0GI IV(5$UQUHIfQdQfHgghik k XhZw{m:Nj  !)F: f||Ce)8F>F)r&ct||}|stjdytd|d|d}ddd}d }|r t j |t |d |vr|ndd }d}|jd k7r/tjd|j|jddy|j} | j dgD]X} | j dd} |dz }|| vs| jjds?tjd| || cS| j d}|rtjd|||y#t$rtjdYywxYw)z;Site-drive equivalent of ``find_quote_workbook`` (DVI-947).z6site_find_quote_workbook: could not resolve site driveNrerfrtrrrrrrhrr-z5site_find_quote_workbook folder fetch failed (%s): %sr.r`rrr(rz:site_find_quote_workbook: found %s after scanning %d itemsrzsite_find_quote_workbook errorzEsite_find_quote_workbook: quote %s not found in %s (scanned %d items))rAr6r7r3r2rr4r5r8rrrrr9r:rN) r%rrrr&r@rkrrrarrrs rsite_find_quote_workbookrsv%UD1H  JK M(7;-z JC&} ~F D  <<\%-@14CvTSUWDF3& Q**DIIdsO=99;D"- xx+ 4'DJJL,A,A',JHHY[_aefK  ((,-C! (KKWk41    MM: ; s+A'E&AE'EE"EE10E1c:t||}|syt|||S)z=Read a file from the SharePoint site drive by path (DVI-947).N)rArl)r%rcr&r@s rsite_drive_read_filers#$UD1H  5(J 77rc`t||}|sytd|d|d} tj|t |d}|j dk7r0t jd|j ||jdd y|jS#t$rt jd YywxYw) uFetch a drive item's raw content by ID from the SharePoint site drive (site-drive equivalent of ``user_drive_read_item`` — DVI-975). Returns the file bytes or None on any failure.Nre/items/rrhr*r-z0site_drive_read_item failed (%s) for item %s: %sr.zsite_drive_read_item error) rAr3r2rr4r5r6r7r8rjr9r:)r%rr&r@rkras rsite_drive_read_itemrs%UD1H  M(77)8 DC ||Ce)3/0/lK F ||Ce)? M J<}]O \C||Ce)   S ! HH]$> ? HHV\ *   12sA DA D 7DD98D9c|xst}td|d|} tj|dd|id}|jdk(S#t $rt jdYy wxYw) z6Delete a draft message by ID. Returns True on success.rorr^r[r)r*zmail_delete_message errorF)rr3r2deleter5r9r6r:)r%rrrkras rmail_delete_messagersw**G M J< @Cs_w>O,PZ\]3&&  12s+AA'&A'i@ item_base_urlrz dict | Nonec  tj|dd|dddddiid }|j d vr/tj d |j |jd dy |jjd}|stj dy t|}d }td|tD]}|||tz}|t|zdz } tj|d|d| d|tt|d|d} | j d vr| j}w| j dk(rtj d| j ||| jd dy |S#t$rtj d Yy wxYw#t$rtj d||Yy wxYw)aUpload bytes to a Graph item path via an upload session (for files >4 MB). ``item_base_url`` is the OneDrive/SharePoint path URL ending with ``:`` but without the trailing ``/content``, e.g. ``https://graph.microsoft.com/v1.0/drives/{id}/root:/{folder}/{file}:``. Returns the completed item JSON (containing ``id`` and ``parentReference``) on success, or ``None`` on error. z/createUploadSessionr[r\r]rz!@microsoft.graph.conflictBehaviorreplacer)rz._upload_via_session: createUploadSession errorNrz8_upload_via_session: createUploadSession failed (%s): %sr. uploadUrlzC_upload_via_session: createUploadSession response missing uploadUrlrr(zbytes -/)z Content-RangezContent-Lengthrrz._upload_via_session: chunk error at byte %d/%drz8_upload_via_session: chunk failed (%s) at byte %d/%d: %sr-)r2rr9r6r:r5r7r8rrlenrange_UPLOAD_CHUNK_SIZErr) r%rr sess_resp upload_urltotal item_datastartchunkendput_resps r_upload_via_sessionrs MMo1 2(/w&7I[\> JK  J. L'')= ?!%%k2J  WX IE#Iq%!34U5#556c%j 1$ ||'-eWAcU!E7%C&)#e*oH   : - I  ! !S (  IIP**E5(--:M O14 U  FG:  MMJESX Y s#&E67F6FF F>=F>)scratch_filename service_user xlsx_bytesrrc@|xs"dtjjd}td|}|rtd|d|d}n2t |}|st jdytd |d |d}t|d z}t|tkDr=t jd ||t|||} | t jd |y tj|dd|dd|d} | j dvr0t jd| j || j"ddy| j%} | j'dd} | j'dij'dd} | r| st jd|y tj&td | d| ddd|id d!} | j d#k7r=t jd$| j || j"ddt)|| | y| j*}t)|| | t jd%|t|d z|S#t$rt jd|YywxYw#t$r&t jd"|t)|| | YywxYw)&uUpload xlsx bytes to a scratch location, convert to PDF via Graph, delete scratch. Follows the B1 recipe proven in DVI-807: 1. Upload xlsx (simple PUT for ≤4 MB; upload session for larger files) 2. GET /drives/{drive_id}/items/{item_id}/content?format=pdf → PDF bytes 3. DELETE the scratch item service_user: when given, use that user's OneDrive for the scratch file (legacy behavior). When omitted (the default), the scratch file goes to the SharePoint site drive (DVI-947) because the pipeline account (togen@) has no personal OneDrive. scratch_filename: filename for the temp xlsx (default: auto-generated UUID name) Returns PDF bytes on success, None on error. winston_scratch_z.xlsxrrorp:zGxlsx_to_pdf: could not resolve SharePoint site drive for scratch uploadNrerfiu1xlsx_to_pdf: %s is %d KB → using upload sessionz)xlsx_to_pdf: upload session failed for %srr[zAapplication/vnd.openxmlformats-officedocument.spreadsheetml.sheetr]rrz xlsx_to_pdf: upload error for %srz*xlsx_to_pdf: upload failed (%s) for %s: %sr.r/rparentReferencedriveIdz9xlsx_to_pdf: upload response missing id or driveId for %srz/content?format=pdfr^T)r+allow_redirectsr,z)xlsx_to_pdf: convert request error for %sr-z+xlsx_to_pdf: convert failed (%s) for %s: %suxlsx_to_pdf: %s → %d KB PDF)uuiduuid4hex_SCRATCH_FOLDERr3rAr6r7r_SIMPLE_UPLOAD_LIMITrrr2rr9r:r5r8rr_delete_drive_itemrj)r%rrrfilename scratch_pathrscratch_drive_idsize_kbrup_resprr@pdf_resp pdf_bytess r xlsx_to_pdfr5s* M%5djjl6F6F5Gu#MH%&az2L&-w|nM,WXY 07 II_ `&-x0@/AVWX *o%G :-- DhPWX'}jI   IIA8 L ll /*'.ug%6[   G   j 0 II<## Tc"   LLN mmD"%G}}.377 2FH ( MxX <<m8H:WWI=P Q$w&78   s" 9   MM$3   5(G4  Iuh0HH ,hI$8NO _  MM$w&78  ]]j ( KKF W^   BHgVsAA BBrcR||tj|jddS)zIPackage raw bytes into a Graph fileAttachment dict for mail_create_draft.ascii)rrr)r b64encodedecode)rrjrs rencode_attachmentrs-#((188A rTakeoffsrfolderctjtjdd}tdttr dtdddnd td t td t d td t}|s tdytd|xst}|stdtdntd|d|dt|||}|td|dd}natdt|d||ddD]5}|jdd}td|jdd|d7ttd t dt|d!d"d#g$}|std%d}nR|jd&d} td'| dd(d)td*t|| r td+n td,|rd-nd.} td/| d0|S)1zRun a two-part verification: (a) list a OneDrive folder, (b) create+delete a draft. Returns True only when both steps succeed. Safe to run in production because the draft is created and immediately deleted. z$%(levelname)s %(name)s %(message)s)levelformatTz#=== Winston Graph verification === z AZURE_CLIENT_ID : Nz...z AZURE_CLIENT_ID : NOT SETz AZURE_TENANT_ID : z WINSTON_MAIL_FROM:  z![1/4] Acquiring app-only token...z FAIL: could not acquire tokenFz OK: token acquired uG[2/4] SKIP: WINSTON_ONEDRIVE_USER not set — cannot verify file accessz< Set WINSTON_ONEDRIVE_USER env var to run this check. z[2/4] Listing 'z' folder for user z FAIL: could not list u4 — check Files.ReadWrite.All permission + user UPNz OK: z item(s) in sizerz - rz (z bytes)z[3/4] Creating test draft in uD[Winston verify] Graph permission check — please ignore and deletez=

Winston Graph verification draft. This can be deleted.

zandy@voyagetech.com)rrru^ FAIL: could not create draft — check Mail.ReadWrite permission + Application Access Policyr/z OK: draft created (id: z...) z[4/4] Deleting test draft...u, OK: draft deleted — verification PASSED uG WARN: draft created but cleanup delete failed — verify it manually PASSEDFAILEDz === Result: z ===)logging basicConfigINFOprintrKrMrrYrrvrrrr) rr!rr%userritr(draftmsg_idrXs r_verifyr4s  gllEG B 01 = r!2 33 7Nkl  -. !"3!4B 78 -.  !E  /0 "#  11D  WX MN x'9$sCD&udF; = +F83gh iB F3u:,l6(; <BQi @vvfb)rvvf~.bg>? @ G )*;) ,- uf - A B \ ]XF L %& Ir__main__zWinston Graph verification) descriptionz--verify store_truezRun permission verification)actionhelpz--onedrive-userz!UPN of the OneDrive owner to test)r9z--folderz+OneDrive folder to list (default: Takeoffs))defaultr9r()N)r-NN)r)zapplication/pdf)Nr )Y__doc__rrr,osr0r pathlibrrOr2 getLoggerr6__file__resolveparentrr rrrenvironrrKrLrM _AZ_AUTHORITYrrrrrRr3r#SITE_DRIVE_UPNr1r tuplefloat__annotations__r$rArYr4listrbbytesrlrqrvrrrrrrrrrrrrrrrrrrrrrrrrrr4__name__argparsesysArgumentParserparser add_argument parse_argsargsverifyexitrr! print_helprrrrUs&   g-.h'')003JJ 4 Q3Q 0"5  4b9  0"5 EO4ZLAUW  '>C ':NQ D[, C 3  dVZIZ   s tDzD7H  s c C D4K sSUT\(scSV[_bf[fL9=/ / /+.:/CF/Dj/,/$J/ /fLP! ! !03!>ADj! D[!HMQ888sTz8UZ]aUa8JNcC$JRWZ^R^,3JN */=@4Z D[N%)%)&*5 5 55s) 5 4Z 5dd"5s)d"59t#5 D[5|%)1 1 11s) 1 4Z 1s)d"1 1n ! !4Z! !  $Z$ !R $ $$$ 4Z $  5#s?d" $NOS s  t W[ '%;s;3;e; ;D$(# f ffDj f * f  T\ fRcS34,C%s[_B3:BcB4BJ z $X $ $1M NF  <>[\ )0ST  J=jk    D {{gd00$++>FAFr