here's usage for curl in Mac

$ curl --help

Usage: curl [options...] <url>

Options: (H) means HTTP/HTTPS only, (F) means FTP only

--anyauth Pick "any" authentication method (H)

-a, --append Append to target file when uploading (F/SFTP)

--basic Use HTTP Basic Authentication (H)

--cacert FILE CA certificate to verify peer against (SSL)

--capath DIR CA directory to verify peer against (SSL)

-E, --cert CERT[:PASSWD] Client certificate file and password (SSL)

--cert-status Verify the status of the server certificate (SSL)

--cert-type TYPE Certificate file type (DER/PEM/ENG) (SSL)

--ciphers LIST SSL ciphers to use (SSL)

--compressed Request compressed response (using deflate or gzip)

-K, --config FILE Read config from FILE

--connect-timeout SECONDS Maximum time allowed for connection

-C, --continue-at OFFSET Resumed transfer OFFSET

-b, --cookie STRING/FILE Read cookies from STRING/FILE (H)

-c, --cookie-jar FILE Write cookies to FILE after operation (H)

--create-dirs Create necessary local directory hierarchy

--crlf Convert LF to CRLF in upload

--crlfile FILE Get a CRL list in PEM format from the given file

-d, --data DATA HTTP POST data (H)

--data-raw DATA HTTP POST data, '@' allowed (H)

--data-ascii DATA HTTP POST ASCII data (H)

--data-binary DATA HTTP POST binary data (H)

--data-urlencode DATA HTTP POST data url encoded (H)

--delegation STRING GSS-API delegation permission

--digest Use HTTP Digest Authentication (H)

--disable-eprt Inhibit using EPRT or LPRT (F)

--disable-epsv Inhibit using EPSV (F)

--dns-servers DNS server addrs to use: 1.1.1.1;2.2.2.2

--dns-interface Interface to use for DNS requests

--dns-ipv4-addr IPv4 address to use for DNS requests, dot notation

--dns-ipv6-addr IPv6 address to use for DNS requests, dot notation

-D, --dump-header FILE Write the headers to FILE

--egd-file FILE EGD socket path for random data (SSL)

--engine ENGINE Crypto engine (use "--engine list" for list) (SSL)

-f, --fail Fail silently (no output at all) on HTTP errors (H)

--false-start Enable TLS False Start.

-F, --form CONTENT Specify HTTP multipart POST data (H)

--form-string STRING Specify HTTP multipart POST data (H)

--ftp-account DATA Account data string (F)

--ftp-alternative-to-user COMMAND String to replace "USER [name]" (F)

--ftp-create-dirs Create the remote dirs if not present (F)

--ftp-method [MULTICWD/NOCWD/SINGLECWD] Control CWD usage (F)

--ftp-pasv Use PASV/EPSV instead of PORT (F)

-P, --ftp-port ADR Use PORT with given address instead of PASV (F)

--ftp-skip-pasv-ip Skip the IP address for PASV (F)

--ftp-pret Send PRET before PASV (for drftpd) (F)

--ftp-ssl-ccc Send CCC after authenticating (F)

--ftp-ssl-ccc-mode ACTIVE/PASSIVE Set CCC mode (F)

--ftp-ssl-control Require SSL/TLS for FTP login, clear for transfer (F)

-G, --get Send the -d data with a HTTP GET (H)

-g, --globoff Disable URL sequences and ranges using {} and []

-H, --header LINE Pass custom header LINE to server (H)

-I, --head Show document info only

-h, --help This help text

--hostpubmd5 MD5 Hex-encoded MD5 string of the host public key. (SSH)

-0, --http1.0 Use HTTP 1.0 (H)

--http1.1 Use HTTP 1.1 (H)

--http2 Use HTTP 2 (H)

--ignore-content-length Ignore the HTTP Content-Length header

-i, --include Include protocol headers in the output (H/F)

-k, --insecure Allow connections to SSL sites without certs (H)

--interface INTERFACE Use network INTERFACE (or address)

-4, --ipv4 Resolve name to IPv4 address

-6, --ipv6 Resolve name to IPv6 address

-j, --junk-session-cookies Ignore session cookies read from file (H)

--keepalive-time SECONDS Wait SECONDS between keepalive probes

--key KEY Private key file name (SSL/SSH)

--key-type TYPE Private key file type (DER/PEM/ENG) (SSL)

--krb LEVEL Enable Kerberos with security LEVEL (F)

--libcurl FILE Dump libcurl equivalent code of this command line

--limit-rate RATE Limit transfer speed to RATE

-l, --list-only List only mode (F/POP3)

--local-port RANGE Force use of RANGE for local port numbers

-L, --location Follow redirects (H)

--location-trusted Like '--location', and send auth to other hosts (H)

--login-options OPTIONS Server login options (IMAP, POP3, SMTP)

-M, --manual Display the full manual

--mail-from FROM Mail from this address (SMTP)

--mail-rcpt TO Mail to this/these addresses (SMTP)

--mail-auth AUTH Originator address of the original email (SMTP)

--max-filesize BYTES Maximum file size to download (H/F)

--max-redirs NUM Maximum number of redirects allowed (H)

-m, --max-time SECONDS Maximum time allowed for the transfer

--metalink Process given URLs as metalink XML file

--negotiate Use HTTP Negotiate (SPNEGO) authentication (H)

-n, --netrc Must read .netrc for user name and password

--netrc-optional Use either .netrc or URL; overrides -n

--netrc-file FILE Specify FILE for netrc

-:, --next Allows the following URL to use a separate set of options

--no-alpn Disable the ALPN TLS extension (H)

-N, --no-buffer Disable buffering of the output stream

--no-keepalive Disable keepalive use on the connection

--no-npn Disable the NPN TLS extension (H)

--no-sessionid Disable SSL session-ID reusing (SSL)

--noproxy List of hosts which do not use proxy

--ntlm Use HTTP NTLM authentication (H)

--oauth2-bearer TOKEN OAuth 2 Bearer Token (IMAP, POP3, SMTP)

-o, --output FILE Write to FILE instead of stdout

--pass PASS Pass phrase for the private key (SSL/SSH)

--path-as-is Do not squash .. sequences in URL path

--pinnedpubkey FILE Public key (PEM/DER) to verify peer against (OpenSSL/GnuTLS/NSS/wolfSSL/CyaSSL/GSKit only)

--post301 Do not switch to GET after following a 301 redirect (H)

--post302 Do not switch to GET after following a 302 redirect (H)

--post303 Do not switch to GET after following a 303 redirect (H)

-#, --progress-bar Display transfer progress as a progress bar

--proto PROTOCOLS Enable/disable PROTOCOLS

--proto-redir PROTOCOLS Enable/disable PROTOCOLS on redirect

-x, --proxy [PROTOCOL://]HOST[:PORT] Use proxy on given port

--proxy-anyauth Pick "any" proxy authentication method (H)

--proxy-basic Use Basic authentication on the proxy (H)

--proxy-digest Use Digest authentication on the proxy (H)

--proxy-negotiate Use HTTP Negotiate (SPNEGO) authentication on the proxy (H)

--proxy-ntlm Use NTLM authentication on the proxy (H)

--proxy-service-name NAME SPNEGO proxy service name

--service-name NAME SPNEGO service name

-U, --proxy-user USER[:PASSWORD] Proxy user and password

--proxy1.0 HOST[:PORT] Use HTTP/1.0 proxy on given port

-p, --proxytunnel Operate through a HTTP proxy tunnel (using CONNECT)

--pubkey KEY Public key file name (SSH)

-Q, --quote CMD Send command(s) to server before transfer (F/SFTP)

--random-file FILE File for reading random data from (SSL)

-r, --range RANGE Retrieve only the bytes within RANGE

--raw Do HTTP "raw"; no transfer decoding (H)

-e, --referer Referer URL (H)

-J, --remote-header-name Use the header-provided filename (H)

-O, --remote-name Write output to a file named as the remote file

--remote-name-all Use the remote file name for all URLs

-R, --remote-time Set the remote file's time on the local output

-X, --request COMMAND Specify request command to use

--resolve HOST:PORT:ADDRESS Force resolve of HOST:PORT to ADDRESS

--retry NUM Retry request NUM times if transient problems occur

--retry-delay SECONDS Wait SECONDS between retries

--retry-max-time SECONDS Retry only within this period

--sasl-ir Enable initial response in SASL authentication

-S, --show-error Show error. With -s, make curl show errors when they occur

-s, --silent Silent mode (don't output anything)

--socks4 HOST[:PORT] SOCKS4 proxy on given host + port

--socks4a HOST[:PORT] SOCKS4a proxy on given host + port

--socks5 HOST[:PORT] SOCKS5 proxy on given host + port

--socks5-hostname HOST[:PORT] SOCKS5 proxy, pass host name to proxy

--socks5-gssapi-service NAME SOCKS5 proxy service name for GSS-API

--socks5-gssapi-nec Compatibility with NEC SOCKS5 server

-Y, --speed-limit RATE Stop transfers below RATE for 'speed-time' secs

-y, --speed-time SECONDS Trigger 'speed-limit' abort after SECONDS (default: 30)

--ssl Try SSL/TLS (FTP, IMAP, POP3, SMTP)

--ssl-reqd Require SSL/TLS (FTP, IMAP, POP3, SMTP)

-2, --sslv2 Use SSLv2 (SSL)

-3, --sslv3 Use SSLv3 (SSL)

--ssl-allow-beast Allow security flaw to improve interop (SSL)

--stderr FILE Where to redirect stderr (use "-" for stdout)

--tcp-nodelay Use the TCP_NODELAY option

-t, --telnet-option OPT=VAL Set telnet option

--tftp-blksize VALUE Set TFTP BLKSIZE option (must be >512)

-z, --time-cond TIME Transfer based on a time condition

-1, --tlsv1 Use => TLSv1 (SSL)

--tlsv1.0 Use TLSv1.0 (SSL)

--tlsv1.1 Use TLSv1.1 (SSL)

--tlsv1.2 Use TLSv1.2 (SSL)

--trace FILE Write a debug trace to FILE

--trace-ascii FILE Like --trace, but without hex output

--trace-time Add time stamps to trace/verbose output

--tr-encoding Request compressed transfer encoding (H)

-T, --upload-file FILE Transfer FILE to destination

--url URL URL to work with

-B, --use-ascii Use ASCII/text transfer

-u, --user USER[:PASSWORD] Server user and password

--tlsuser USER TLS username

--tlspassword STRING TLS password

--tlsauthtype STRING TLS authentication type (default: SRP)

--unix-socket FILE Connect through this Unix domain socket

-A, --user-agent STRING Send User-Agent STRING to server (H)

-v, --verbose Make the operation more talkative

-V, --version Show version number and quit

-w, --write-out FORMAT Use output FORMAT after completion

--xattr Store metadata in extended file attributes

-q Disable .curlrc (must be first parameter)

hmm, seems to just hang there... I CTRL-C after 30s.. Should I wait longer?

$ /Users/Telecart/Downloads/Ubooquity/PageBuilder/imageGet.sh --comic "Action Comics" -r

Comic: Action Comics

Result:

Getting series ID

^C

...

This command does seem to bring back the SERP, so I don't think it's the wget itself...

wget -e robots=off --user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/601.6.17 (KHTML, like Gecko) Version/9.1.1 Safari/601.6.17" -qO - "https://www.google.com/search?client=safari&rls=en&q='action+comics'+inurl:series+intitle:Digital&ie=UTF-8&oe=UTF=8"

'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109....'action comics' inurl:series intitle:Digital - Google Search (function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109<!doctype html><html itemscope="" itemtype="http://schema.org/SearchResultsPage" lang="en"><head><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop="image"><link href="/images/branding/product/ico/googleg_lodp.ico" rel="shortcut icon"><meta content="origin" id="mref" name="referrer"><title>'action comics' inurl:series intitle:Digital - Google Search</title> <script>(function(){window.google={kEI:'EJD5V6WcOIHne5qzubgE',kEXPI:'20782,750721,1351633,1351949,1352019,3700250,3700400,4029370,4029815,4031109

...

[I cut it off as the response is quite long]

usage patterns look similar to me. I See imageGet.sh also uses the sed command - perhaps it's a similar issue as before?

wget --help

GNU Wget 1.16.3, a non-interactive network retriever.

Usage: wget [OPTION]... [URL]...

Mandatory arguments to long options are mandatory for short options too.

Startup:

-V, --version display the version of Wget and exit

-h, --help print this help

-b, --background go to background after startup

-e, --execute=COMMAND execute a `.wgetrc'-style command

Logging and input file:

-o, --output-file=FILE log messages to FILE

-a, --append-output=FILE append messages to FILE

-d, --debug print lots of debugging information

-q, --quiet quiet (no output)

-v, --verbose be verbose (this is the default)

-nv, --no-verbose turn off verboseness, without being quiet

--report-speed=TYPE output bandwidth as TYPE. TYPE can be bits

-i, --input-file=FILE download URLs found in local or external FILE

-F, --force-html treat input file as HTML

-B, --base=URL resolves HTML input-file links (-i -F)

relative to URL

--config=FILE specify config file to use

--no-config do not read any config file

Download:

-t, --tries=NUMBER set number of retries to NUMBER (0 unlimits)

--retry-connrefused retry even if connection is refused

-O, --output-document=FILE write documents to FILE

-nc, --no-clobber skip downloads that would download to

existing files (overwriting them)

-c, --continue resume getting a partially-downloaded file

--start-pos=OFFSET start downloading from zero-based position OFFSET

--progress=TYPE select progress gauge type

--show-progress display the progress bar in any verbosity mode

-N, --timestamping don't re-retrieve files unless newer than

local

--no-use-server-timestamps don't set the local file's timestamp by

the one on the server

-S, --server-response print server response

--spider don't download anything

-T, --timeout=SECONDS set all timeout values to SECONDS

--dns-timeout=SECS set the DNS lookup timeout to SECS

--connect-timeout=SECS set the connect timeout to SECS

--read-timeout=SECS set the read timeout to SECS

-w, --wait=SECONDS wait SECONDS between retrievals

--waitretry=SECONDS wait 1..SECONDS between retries of a retrieval

--random-wait wait from 0.5*WAIT...1.5*WAIT secs between retrievals

--no-proxy explicitly turn off proxy

-Q, --quota=NUMBER set retrieval quota to NUMBER

--bind-address=ADDRESS bind to ADDRESS (hostname or IP) on local host

--limit-rate=RATE limit download rate to RATE

--no-dns-cache disable caching DNS lookups

--restrict-file-names=OS restrict chars in file names to ones OS allows

--ignore-case ignore case when matching files/directories

-4, --inet4-only connect only to IPv4 addresses

-6, --inet6-only connect only to IPv6 addresses

--prefer-family=FAMILY connect first to addresses of specified family,

one of IPv6, IPv4, or none

--user=USER set both ftp and http user to USER

--password=PASS set both ftp and http password to PASS

--ask-password prompt for passwords

--no-iri turn off IRI support

--local-encoding=ENC use ENC as the local encoding for IRIs

--remote-encoding=ENC use ENC as the default remote encoding

--unlink remove file before clobber

Directories:

-nd, --no-directories don't create directories

-x, --force-directories force creation of directories

-nH, --no-host-directories don't create host directories

--protocol-directories use protocol name in directories

-P, --directory-prefix=PREFIX save files to PREFIX/..

--cut-dirs=NUMBER ignore NUMBER remote directory components

HTTP options:

--http-user=USER set http user to USER

--http-password=PASS set http password to PASS

--no-cache disallow server-cached data

--default-page=NAME change the default page name (normally

this is 'index.html'.)

-E, --adjust-extension save HTML/CSS documents with proper extensions

--ignore-length ignore 'Content-Length' header field

--header=STRING insert STRING among the headers

--max-redirect maximum redirections allowed per page

--proxy-user=USER set USER as proxy username

--proxy-password=PASS set PASS as proxy password

--referer=URL include 'Referer: URL' header in HTTP request

--save-headers save the HTTP headers to file

-U, --user-agent=AGENT identify as AGENT instead of Wget/VERSION

--no-http-keep-alive disable HTTP keep-alive (persistent connections)

--no-cookies don't use cookies

--load-cookies=FILE load cookies from FILE before session

--save-cookies=FILE save cookies to FILE after session

--keep-session-cookies load and save session (non-permanent) cookies

--post-data=STRING use the POST method; send STRING as the data

--post-file=FILE use the POST method; send contents of FILE

--method=HTTPMethod use method "HTTPMethod" in the request

--body-data=STRING send STRING as data. --method MUST be set

--body-file=FILE send contents of FILE. --method MUST be set

--content-disposition honor the Content-Disposition header when

choosing local file names (EXPERIMENTAL)

--content-on-error output the received content on server errors

--auth-no-challenge send Basic HTTP authentication information

without first waiting for the server's

challenge

HTTPS (SSL/TLS) options:

--secure-protocol=PR choose secure protocol, one of auto, SSLv2,

SSLv3, TLSv1 and PFS

--https-only only follow secure HTTPS links

--no-check-certificate don't validate the server's certificate

--certificate=FILE client certificate file

--certificate-type=TYPE client certificate type, PEM or DER

--private-key=FILE private key file

--private-key-type=TYPE private key type, PEM or DER

--ca-certificate=FILE file with the bundle of CAs

--ca-directory=DIR directory where hash list of CAs is stored

--crl-file=FILE file with bundle of CRLs

--random-file=FILE file with random data for seeding the SSL PRNG

--egd-file=FILE file naming the EGD socket with random data

FTP options:

--ftp-user=USER set ftp user to USER

--ftp-password=PASS set ftp password to PASS

--no-remove-listing don't remove '.listing' files

--no-glob turn off FTP file name globbing

--no-passive-ftp disable the "passive" transfer mode

--preserve-permissions preserve remote file permissions

--retr-symlinks when recursing, get linked-to files (not dir)

WARC options:

--warc-file=FILENAME save request/response data to a .warc.gz file

--warc-header=STRING insert STRING into the warcinfo record

--warc-max-size=NUMBER set maximum size of WARC files to NUMBER

--warc-cdx write CDX index files

--warc-dedup=FILENAME do not store records listed in this CDX file

--no-warc-compression do not compress WARC files with GZIP

--no-warc-digests do not calculate SHA1 digests

--no-warc-keep-log do not store the log file in a WARC record

--warc-tempdir=DIRECTORY location for temporary files created by the

WARC writer

Recursive download:

-r, --recursive specify recursive download

-l, --level=NUMBER maximum recursion depth (inf or 0 for infinite)

--delete-after delete files locally after downloading them

-k, --convert-links make links in downloaded HTML or CSS point to

local files

--backups=N before writing file X, rotate up to N backup files

-K, --backup-converted before converting file X, back up as X.orig

-m, --mirror shortcut for -N -r -l inf --no-remove-listing

-p, --page-requisites get all images, etc. needed to display HTML page

--strict-comments turn on strict (SGML) handling of HTML comments

Recursive accept/reject:

-A, --accept=LIST comma-separated list of accepted extensions

-R, --reject=LIST comma-separated list of rejected extensions

--accept-regex=REGEX regex matching accepted URLs

--reject-regex=REGEX regex matching rejected URLs

--regex-type=TYPE regex type (posix)

-D, --domains=LIST comma-separated list of accepted domains

--exclude-domains=LIST comma-separated list of rejected domains

--follow-ftp follow FTP links from HTML documents

--follow-tags=LIST comma-separated list of followed HTML tags

--ignore-tags=LIST comma-separated list of ignored HTML tags

-H, --span-hosts go to foreign hosts when recursive

-L, --relative follow relative links only

-I, --include-directories=LIST list of allowed directories

--trust-server-names use the name specified by the redirection

URL's last component

-X, --exclude-directories=LIST list of excluded directories

-np, --no-parent don't ascend to the parent directory

I assume it's supposed to create the file in the same dir as imageGet.sh right?

$ /Users/Telecart/Downloads/Ubooquity/PageBuilder/imageGet.sh -c "action comics" -d 2016

Comic: action comics

Date: 2016

Downloading image

action%20comics

but there's no files created anywhere as far as I can tell...

tried it with -r, not giving different result...

$ /Users/Telecart/Downloads/Ubooquity/PageBuilder/imageGet.sh --comic "Action Comics" --date "2016" -r

Comic: Action Comics

Date: 2016

Result:

Downloading image

Action%20Comics

Btw, books.css I think got left behind at some point and now shows default theme look & feel. I just copied over it from old folder and it looks more in-line.

...which is also why page 2 for marvel looks different than page 1!

mostly works - files being generated with data, except can't get any images

No matter what I consistently get mv: rename /Path/To/Name of Comics 2016.jpg to /Path/To/Name of Comics (2016)/folder.jpg: No such file or directory

Done!

Any thoughts?

great! Will update when I get a chance to test it out!

Other Mac users may still need to install wget I assume (and to do that, need homebrew and OpenSSL)

Otherwise no reason this wouldn't work on a Mac

oh, derp, that'll do it.